2024-09-15T23:40:06.329 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T23:40:06.355 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:40:06.457 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906345' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsLGtpLr6Bh57v220uF9FYfww0UN+Yg2OlPIqhvLs/uJC51yNCg3uLLRQ/NzbunQRcQZWwlTJtmoaNe0dOlGQY= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGmCPcjFTHNFILDny1BtPfLOOVfd4bpbcFuq9+Wl8h1t/eLEBxWpifH1zsG86163kOnZaOc8Wd8jNJhW68SWnoA= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGMjG/dWCGIlqIujEJ+HekNHDJmzFB01YJoJ6MhMvtYvMHIpxfAitDrWzeArt9eqpLio2CAhrgP37q3n9eFClXo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T23:40:06.458 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T23:40:06.459 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T23:40:06.459 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T23:40:06.460 INFO:teuthology.task.internal:Checking packages... 2024-09-15T23:40:06.482 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T23:40:06.482 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T23:40:06.482 INFO:teuthology.packaging:ref: None 2024-09-15T23:40:06.483 INFO:teuthology.packaging:tag: None 2024-09-15T23:40:06.483 INFO:teuthology.packaging:branch: squid 2024-09-15T23:40:06.483 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:40:06.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T23:40:06.719 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T23:40:06.721 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T23:40:06.728 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T23:40:06.728 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T23:40:06.741 INFO:teuthology.task.internal:Saving configuration 2024-09-15T23:40:06.760 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T23:40:06.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T23:40:06.790 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:35:12.043951', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsLGtpLr6Bh57v220uF9FYfww0UN+Yg2OlPIqhvLs/uJC51yNCg3uLLRQ/NzbunQRcQZWwlTJtmoaNe0dOlGQY='} 2024-09-15T23:40:06.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:35:12.040560', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGmCPcjFTHNFILDny1BtPfLOOVfd4bpbcFuq9+Wl8h1t/eLEBxWpifH1zsG86163kOnZaOc8Wd8jNJhW68SWnoA='} 2024-09-15T23:40:06.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:35:12.042448', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGMjG/dWCGIlqIujEJ+HekNHDJmzFB01YJoJ6MhMvtYvMHIpxfAitDrWzeArt9eqpLio2CAhrgP37q3n9eFClXo='} 2024-09-15T23:40:06.834 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T23:40:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T23:40:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T23:40:06.841 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T23:40:06.841 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T23:40:06.920 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcdbfccfbe0>, signals=[15]) 2024-09-15T23:40:06.920 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T23:40:06.926 INFO:teuthology.task.internal:Opening connections... 2024-09-15T23:40:06.927 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-15T23:40:06.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:40:07.005 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-09-15T23:40:07.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:40:07.083 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-15T23:40:07.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:40:07.160 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T23:40:07.168 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-15T23:40:07.186 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-15T23:40:07.186 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:40:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:40:07.245 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:40:07.245 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:40:07.245 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:40:07.245 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:40:07.245 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-15T23:40:07.268 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-09-15T23:40:07.287 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-09-15T23:40:07.287 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-09-15T23:40:07.344 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-09-15T23:40:07.344 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-09-15T23:40:07.344 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-09-15T23:40:07.344 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:40:07.344 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:40:07.345 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:40:07.346 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-09-15T23:40:07.368 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-15T23:40:07.386 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-15T23:40:07.386 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-15T23:40:07.443 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:40:07.444 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:40:07.444 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-15T23:40:07.574 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T23:40:07.584 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T23:40:07.590 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T23:40:07.590 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:40:07.593 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:40:07.595 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:40:07.612 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T23:40:07.618 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T23:40:07.618 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:40:07.650 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:40:07.653 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:40:07.772 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T23:40:07.847 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T23:40:07.864 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T23:40:07.865 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T23:40:07.865 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:40:07.866 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:40:07.866 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:40:07.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:07.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:07.885 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-09-15T23:40:07.885 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-15T23:40:07.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:07.886 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-15T23:40:07.942 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:40:07.942 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-503.el9.x86_64 2024-09-15T23:40:07.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-09-15T23:40:07.943 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:40:07.944 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-15T23:40:07.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-15T23:40:07.945 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:40:07.945 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-503.el9.x86_64 2024-09-15T23:40:07.945 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-09-15T23:40:10.472 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-15T23:40:10.527 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-15T23:40:11.366 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-15T23:40:12.265 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-15T23:40:13.678 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-15T23:40:18.129 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-15T23:40:18.715 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-15T23:40:19.488 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 118 kB/s | 19 kB 00:00 2024-09-15T23:40:20.058 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-15T23:40:20.232 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-15T23:40:20.443 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-15T23:40:21.378 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-15T23:40:24.334 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T23:40:25.768 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-15T23:40:25.965 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T23:40:26.672 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-15T23:40:27.025 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-15T23:40:28.030 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:40:28.030 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:40:28.137 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-15T23:40:28.147 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-15T23:40:28.147 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-09-15T23:40:28.147 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-15T23:40:28.147 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-09-15T23:40:28.148 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:40:28.149 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 79 M 2024-09-15T23:40:28.149 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 126 M 2024-09-15T23:40:28.149 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-15T23:40:28.258 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 257 kB/s | 23 kB 00:00 2024-09-15T23:40:29.123 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:40:29.123 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:40:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-15T23:40:29.232 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-15T23:40:29.232 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-15T23:40:29.232 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-15T23:40:29.232 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-15T23:40:29.232 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-15T23:40:29.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:40:29.234 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-15T23:40:29.234 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-15T23:40:29.234 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-15T23:40:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 283 kB/s | 23 kB 00:00 2024-09-15T23:40:30.869 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-15T23:40:32.443 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.9 MB/s | 16 MB 00:03 2024-09-15T23:40:32.717 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.4 MB/s | 29 MB 00:04 2024-09-15T23:40:33.286 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 1.4 kB/s | 1.7 kB 00:01 2024-09-15T23:40:33.593 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.7 MB/s | 29 MB 00:04 2024-09-15T23:40:35.327 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:40:35.327 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:40:35.431 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-15T23:40:35.439 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-15T23:40:35.439 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-15T23:40:35.439 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-15T23:40:35.439 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-15T23:40:35.440 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:40:35.440 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-15T23:40:35.440 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:40:35.440 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:40:35.440 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 79 M 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 126 M 2024-09-15T23:40:35.441 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-15T23:40:35.639 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 265 kB/s | 23 kB 00:00 2024-09-15T23:40:36.054 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.1 MB/s | 35 MB 00:06 2024-09-15T23:40:36.054 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:40:36.054 INFO:teuthology.orchestra.run.smithi031.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-15T23:40:36.425 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.2 MB/s | 35 MB 00:08 2024-09-15T23:40:36.425 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:40:36.425 INFO:teuthology.orchestra.run.smithi081.stdout:Total 9.5 MB/s | 79 MB 00:08 2024-09-15T23:40:36.770 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-15T23:40:37.108 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-15T23:40:37.146 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-15T23:40:37.146 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-15T23:40:37.472 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-15T23:40:37.472 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-15T23:40:38.141 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-15T23:40:38.142 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-15T23:40:38.430 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-15T23:40:38.430 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-15T23:40:39.488 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:03 2024-09-15T23:40:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-15T23:40:40.870 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-15T23:40:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:40:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:41.634 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:40:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:43.046 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.9 MB/s | 29 MB 00:07 2024-09-15T23:40:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:44.413 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-15T23:40:44.414 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:40:44.414 INFO:teuthology.orchestra.run.smithi102.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-15T23:40:44.512 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:44.639 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:45.102 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-15T23:40:45.449 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-15T23:40:45.449 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-15T23:40:46.389 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-15T23:40:46.390 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-15T23:40:48.826 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-15T23:40:49.448 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:49.556 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:40:49.572 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:40:49.656 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:40:52.445 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:40:52.577 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:54.442 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:40:54.915 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:40:57.418 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:40:57.452 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:02.324 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:32.765 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:32.810 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:33.132 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:33.132 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:41:33.132 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:41:33.132 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:41:34.025 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:34.025 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:41:34.025 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:41:34.026 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-15T23:41:34.208 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:34.351 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:34.430 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.431 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:34.431 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:41:34.431 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:41:34.431 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:41:34.431 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T23:41:34.431 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:41:34.432 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:34.489 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:34.490 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:34.490 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-09-15T23:41:34.525 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-09-15T23:41:34.525 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-09-15T23:41:34.525 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 82.el9 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8274670 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:41:34.526 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:41:34.527 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:41:34.528 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:41:34.528 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:41:34.528 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:41:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:41:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:41:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:41:35.201 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:41:35.417 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-15T23:41:35.748 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:35.891 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:41:35.891 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:41:35.891 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:41:35.891 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T23:41:35.891 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:41:35.891 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:35.950 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:35.950 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:35.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-15T23:41:35.985 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-15T23:41:35.985 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-15T23:41:35.985 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:41:35.986 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:41:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:41:35.988 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:41:35.988 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:41:35.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:41:36.543 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:41:36.571 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-09-15T23:41:36.573 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-09-15T23:41:36.589 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.VSqVW7sKNt 2024-09-15T23:41:36.589 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VSqVW7sKNt 2024-09-15T23:41:36.650 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-15T23:41:36.655 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.VSqVW7sKNt 2024-09-15T23:41:36.791 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.VSqVW7sKNt is 6KB 2024-09-15T23:41:36.839 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.VSqVW7sKNt 2024-09-15T23:41:36.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:41:36.924 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:41:36.924 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:41:36.924 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:41:36.924 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:41:36.926 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:41:37.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-09-15T23:41:37.984 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:41:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-15T23:41:38.007 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-15T23:41:38.022 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.AdWf0hgrY1 2024-09-15T23:41:38.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AdWf0hgrY1 2024-09-15T23:41:38.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.AdWf0hgrY1 2024-09-15T23:41:38.225 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.AdWf0hgrY1 is 6KB 2024-09-15T23:41:38.273 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.AdWf0hgrY1 2024-09-15T23:41:38.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:41:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:41:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:41:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:41:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:41:38.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:41:38.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-15T23:41:40.591 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:40.636 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:41:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:41:41.017 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:41:41.853 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-15T23:41:42.243 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:42.317 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:42.317 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:42.317 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:41:42.317 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:41:42.317 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:41:42.317 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-15T23:41:42.317 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:41:42.317 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:41:42.375 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:41:42.376 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:41:42.376 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 82.el9 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-09-15T23:41:42.411 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8274670 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:41:42.412 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:41:42.413 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:41:42.413 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-09-15T23:41:42.413 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:41:42.413 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:41:42.414 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:41:42.414 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:41:42.414 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:41:43.110 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-09-15T23:41:44.322 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:41:44.351 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-09-15T23:41:44.353 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-15T23:41:44.368 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.K82tVF3j7W 2024-09-15T23:41:44.369 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.K82tVF3j7W 2024-09-15T23:41:44.435 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.K82tVF3j7W 2024-09-15T23:41:44.580 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.K82tVF3j7W is 6KB 2024-09-15T23:41:44.628 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.K82tVF3j7W 2024-09-15T23:41:44.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:41:44.705 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:41:44.705 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:41:44.705 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:41:44.705 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:41:44.707 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:41:44.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-09-15T23:42:07.109 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T23:42:07.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:42:07.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:08.513 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T23:42:08.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:42:08.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:14.878 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-15T23:42:14.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:42:14.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:25.601 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-09-15T23:42:26.881 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-15T23:42:33.281 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-15T23:42:34.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:42:34.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:35.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:42:35.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:42.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:42:42.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:42:45.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-15T23:42:54.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-15T23:42:57.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:42:57.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:43:06.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:43:06.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:43:07.111 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-15T23:43:07.596 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:43:07.596 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:43:07.596 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-15T23:43:07.611 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:43:07.611 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:43:07.611 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:43:07.612 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:43:08.613 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T23:43:08.613 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T23:43:08.684 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-15T23:43:08.716 DEBUG:teuthology.parallel:result is None 2024-09-15T23:43:29.222 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-15T23:43:29.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:43:29.705 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:43:29.706 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-15T23:43:29.721 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:43:29.721 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:43:29.721 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:43:29.721 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:43:30.722 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-15T23:43:30.722 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T23:43:30.772 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-09-15T23:43:30.795 DEBUG:teuthology.parallel:result is None 2024-09-15T23:43:34.607 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T23:43:46.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:43:46.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:43:47.253 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-09-15T23:43:47.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:43:47.603 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:43:47.603 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-09-15T23:43:47.619 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:43:47.619 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:43:47.619 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:43:47.619 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:43:48.620 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T23:43:48.620 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T23:43:48.694 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-09-15T23:43:48.713 DEBUG:teuthology.parallel:result is None 2024-09-15T23:43:48.713 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T23:43:48.720 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T23:43:48.721 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:43:48.725 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:43:48.727 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:43:48.746 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T23:43:48.753 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T23:43:48.761 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T23:43:48.761 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:43:48.785 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:43:48.788 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:43:48.889 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T23:43:48.895 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T23:43:48.896 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:43:48.899 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:43:48.901 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:43:48.930 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.932 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.936 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.942 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.944 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.947 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:43:48.948 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T23:43:48.955 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T23:43:48.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:43:48.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:43:48.989 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:43:49.015 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T23:43:49.024 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T23:43:49.024 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:43:49.057 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:43:49.061 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:43:49.077 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.205 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.333 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:43:49.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:43:49.400 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.439 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.495 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.573 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.622 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:43:49.622 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:43:49.689 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.732 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:43:49.790 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.875 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:43:49.920 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:43:49.920 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:43:49.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-15T23:43:49.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-09-15T23:43:49.994 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-15T23:43:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:43:50.039 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:43:50.082 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:43:50.595 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T23:43:50.602 INFO:teuthology.task.internal:Starting timer... 2024-09-15T23:43:50.602 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T23:43:50.612 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T23:43:50.620 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T23:43:50.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-15T23:43:50.687 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-15T23:43:50.689 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-09-15T23:43:50.752 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-09-15T23:43:50.754 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-15T23:43:50.825 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-15T23:43:50.827 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T23:43:50.827 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-15T23:43:50.848 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-15T23:43:50.849 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-09-15T23:43:50.872 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-09-15T23:43:50.873 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-15T23:43:50.900 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-15T23:43:50.900 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-09-15T23:43:50.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:43:50.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:43:50.930 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:43:50.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:43:50.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:43:50.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:43:50.989 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T23:43:50.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:43:51.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:43:51.054 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:43:51.091 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T23:43:51.102 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T23:43:51.115 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T23:43:51.116 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi081.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T23:47:38.561 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-09-15T23:47:38.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:47:38.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:47:38.643 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-15T23:47:38.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-15T23:47:38.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:47:38.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:47:38.809 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-09-15T23:47:38.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-15T23:47:38.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:47:38.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:47:38.974 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-15T23:47:39.057 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-15T23:47:39.058 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T23:47:39.070 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T23:47:39.071 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T23:47:39.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:47:39.074 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T23:47:39.074 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:47:39.077 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T23:47:39.077 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:47:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T23:47:39.108 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T23:47:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T23:47:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T23:47:39.127 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T23:47:39.142 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T23:47:39.151 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-09-15T23:47:39.151 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-15T23:47:39.163 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-09-15T23:47:39.163 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-15T23:47:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-15T23:47:39.178 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-15T23:47:39.178 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T23:47:39.179 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T23:47:39.193 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T23:47:39.193 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T23:47:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T23:47:39.206 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T23:47:39.250 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:47:39.253 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:47:39.253 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-15T23:47:39.253 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.253 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.253 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.254 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.256 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:47:39.260 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:47:39.261 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-15T23:47:39.261 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.261 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.261 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.261 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.262 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.266 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:47:39.267 INFO:teuthology.run_tasks:Running task install... 2024-09-15T23:47:39.276 DEBUG:teuthology.task.install:project ceph 2024-09-15T23:47:39.276 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T23:47:39.276 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T23:47:39.276 INFO:teuthology.task.install:Using flavor: default 2024-09-15T23:47:39.282 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T23:47:39.282 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T23:47:39.282 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T23:47:39.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:47:39.284 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T23:47:39.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:47:39.285 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T23:47:39.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:47:39.427 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T23:47:39.427 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T23:47:39.440 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T23:47:39.440 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T23:47:39.444 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T23:47:39.444 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T23:47:39.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T23:47:39.537 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:47:39.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T23:47:39.549 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T23:47:39.549 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:47:39.549 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T23:47:39.556 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T23:47:39.556 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:47:39.556 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T23:47:39.566 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T23:47:39.566 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T23:47:39.575 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T23:47:39.575 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T23:47:39.582 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T23:47:39.582 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T23:47:39.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T23:47:39.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T23:47:39.647 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T23:47:39.711 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T23:47:39.716 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T23:47:39.721 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T23:47:39.778 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-15T23:47:39.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-15T23:47:39.787 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-09-15T23:47:39.788 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-15T23:47:39.789 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-09-15T23:47:39.790 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-15T23:47:40.019 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-15T23:47:40.022 INFO:teuthology.orchestra.run.smithi081.stdout:45 files removed 2024-09-15T23:47:40.035 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-15T23:47:40.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T23:47:40.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T23:47:40.062 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T23:47:40.603 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 310 kB/s | 83 kB 00:00 2024-09-15T23:47:40.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 320 kB/s | 83 kB 00:00 2024-09-15T23:47:40.629 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 298 kB/s | 83 kB 00:00 2024-09-15T23:47:40.887 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T23:47:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T23:47:40.916 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-09-15T23:47:41.162 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T23:47:41.183 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-15T23:47:41.188 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-15T23:47:41.858 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-15T23:47:42.971 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-15T23:47:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-15T23:47:43.643 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-15T23:47:44.815 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-15T23:47:44.889 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-15T23:47:49.203 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-15T23:47:50.240 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.5 MB 00:01 2024-09-15T23:47:50.348 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-15T23:47:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 135 kB/s | 19 kB 00:00 2024-09-15T23:47:51.583 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-15T23:47:51.775 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-15T23:47:51.866 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-15T23:47:52.754 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-15T23:47:52.812 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T23:47:57.005 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T23:47:58.243 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T23:47:58.426 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-15T23:47:59.335 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:47:59.335 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:47:59.349 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T23:47:59.350 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T23:47:59.351 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T23:47:59.352 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T23:47:59.353 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T23:47:59.356 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T23:47:59.357 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T23:47:59.419 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-15T23:47:59.433 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-15T23:47:59.433 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-15T23:47:59.433 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-15T23:47:59.433 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-15T23:47:59.433 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T23:47:59.434 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T23:47:59.435 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T23:47:59.436 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T23:47:59.437 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T23:47:59.438 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T23:47:59.439 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T23:47:59.440 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T23:47:59.441 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T23:47:59.442 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T23:47:59.443 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T23:47:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T23:47:59.445 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-09-15T23:47:59.446 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 8 Packages 2024-09-15T23:47:59.447 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:47:59.447 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 226 M 2024-09-15T23:47:59.447 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-15T23:48:00.215 INFO:teuthology.orchestra.run.smithi102.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-15T23:48:00.440 INFO:teuthology.orchestra.run.smithi102.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-15T23:48:00.574 INFO:teuthology.orchestra.run.smithi102.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-15T23:48:00.588 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.589 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.599 INFO:teuthology.orchestra.run.smithi102.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 915 kB/s | 145 kB 00:00 2024-09-15T23:48:00.602 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T23:48:00.603 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.604 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T23:48:00.605 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.607 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T23:48:00.610 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.611 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.673 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-15T23:48:00.691 INFO:teuthology.orchestra.run.smithi102.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 20 MB/s | 2.4 MB 00:00 2024-09-15T23:48:00.691 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-15T23:48:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T23:48:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T23:48:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T23:48:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T23:48:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T23:48:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T23:48:00.697 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T23:48:00.698 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T23:48:00.699 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T23:48:00.700 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T23:48:00.701 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T23:48:00.702 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T23:48:00.703 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-09-15T23:48:00.704 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:Install 148 Packages 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 8 Packages 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 226 M 2024-09-15T23:48:00.705 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-15T23:48:00.733 INFO:teuthology.orchestra.run.smithi102.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-15T23:48:00.837 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.838 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.850 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T23:48:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T23:48:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T23:48:00.852 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.854 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T23:48:00.856 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.857 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T23:48:00.923 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-15T23:48:00.934 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-15T23:48:00.934 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-15T23:48:00.934 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-15T23:48:00.934 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-15T23:48:00.934 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T23:48:00.935 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T23:48:00.936 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T23:48:00.937 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T23:48:00.938 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T23:48:00.939 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T23:48:00.940 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T23:48:00.940 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T23:48:00.940 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T23:48:00.940 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T23:48:00.940 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T23:48:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T23:48:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T23:48:00.943 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T23:48:00.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T23:48:00.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T23:48:00.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T23:48:00.947 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-15T23:48:00.948 INFO:teuthology.orchestra.run.smithi031.stdout:Install 148 Packages 2024-09-15T23:48:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 8 Packages 2024-09-15T23:48:00.949 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 226 M 2024-09-15T23:48:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-15T23:48:01.025 INFO:teuthology.orchestra.run.smithi102.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-15T23:48:01.284 INFO:teuthology.orchestra.run.smithi102.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-15T23:48:01.334 INFO:teuthology.orchestra.run.smithi102.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 507 kB/s | 25 kB 00:00 2024-09-15T23:48:01.569 INFO:teuthology.orchestra.run.smithi081.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-09-15T23:48:01.776 INFO:teuthology.orchestra.run.smithi102.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 14 MB/s | 11 MB 00:00 2024-09-15T23:48:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.1 MB/s | 228 kB 00:00 2024-09-15T23:48:01.960 INFO:teuthology.orchestra.run.smithi102.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 361 kB/s | 27 kB 00:00 2024-09-15T23:48:02.010 INFO:teuthology.orchestra.run.smithi102.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 634 kB/s | 32 kB 00:00 2024-09-15T23:48:02.011 INFO:teuthology.orchestra.run.smithi081.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 8.2 MB/s | 5.4 MB 00:00 2024-09-15T23:48:02.036 INFO:teuthology.orchestra.run.smithi081.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.9 MB/s | 896 kB 00:00 2024-09-15T23:48:02.094 INFO:teuthology.orchestra.run.smithi031.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-09-15T23:48:02.095 INFO:teuthology.orchestra.run.smithi081.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.7 MB/s | 145 kB 00:00 2024-09-15T23:48:02.194 INFO:teuthology.orchestra.run.smithi102.stdout:(14/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 17 MB 00:01 2024-09-15T23:48:02.227 INFO:teuthology.orchestra.run.smithi102.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 3.4 MB/s | 751 kB 00:00 2024-09-15T23:48:02.270 INFO:teuthology.orchestra.run.smithi081.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-15T23:48:02.277 INFO:teuthology.orchestra.run.smithi102.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.9 MB/s | 163 kB 00:00 2024-09-15T23:48:02.311 INFO:teuthology.orchestra.run.smithi102.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.5 MB/s | 127 kB 00:00 2024-09-15T23:48:02.378 INFO:teuthology.orchestra.run.smithi102.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 4.9 MB/s | 506 kB 00:00 2024-09-15T23:48:02.478 INFO:teuthology.orchestra.run.smithi102.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 451 kB/s | 45 kB 00:00 2024-09-15T23:48:02.562 INFO:teuthology.orchestra.run.smithi102.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T23:48:02.621 INFO:teuthology.orchestra.run.smithi081.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-09-15T23:48:02.646 INFO:teuthology.orchestra.run.smithi102.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-15T23:48:02.729 INFO:teuthology.orchestra.run.smithi102.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T23:48:02.811 INFO:teuthology.orchestra.run.smithi031.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 896 kB 00:00 2024-09-15T23:48:02.813 INFO:teuthology.orchestra.run.smithi102.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.5 MB/s | 300 kB 00:00 2024-09-15T23:48:02.877 INFO:teuthology.orchestra.run.smithi031.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-15T23:48:02.879 INFO:teuthology.orchestra.run.smithi081.stdout:(7/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 21 MB 00:01 2024-09-15T23:48:02.880 INFO:teuthology.orchestra.run.smithi102.stdout:(24/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 9.4 MB/s | 5.3 MB 00:00 2024-09-15T23:48:02.905 INFO:teuthology.orchestra.run.smithi102.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.1 MB/s | 99 kB 00:00 2024-09-15T23:48:02.948 INFO:teuthology.orchestra.run.smithi081.stdout:(8/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 7.0 MB/s | 4.7 MB 00:00 2024-09-15T23:48:02.955 INFO:teuthology.orchestra.run.smithi102.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T23:48:03.030 INFO:teuthology.orchestra.run.smithi081.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 303 kB/s | 25 kB 00:00 2024-09-15T23:48:03.047 INFO:teuthology.orchestra.run.smithi102.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-15T23:48:03.148 INFO:teuthology.orchestra.run.smithi102.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 319 kB/s | 32 kB 00:00 2024-09-15T23:48:03.256 INFO:teuthology.orchestra.run.smithi102.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.3 MB/s | 146 kB 00:00 2024-09-15T23:48:03.332 INFO:teuthology.orchestra.run.smithi102.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 7.3 MB/s | 3.1 MB 00:00 2024-09-15T23:48:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 14 MB/s | 11 MB 00:00 2024-09-15T23:48:03.722 INFO:teuthology.orchestra.run.smithi081.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.4 MB/s | 228 kB 00:00 2024-09-15T23:48:03.765 INFO:teuthology.orchestra.run.smithi102.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 7.0 MB/s | 3.6 MB 00:00 2024-09-15T23:48:03.789 INFO:teuthology.orchestra.run.smithi081.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 405 kB/s | 27 kB 00:00 2024-09-15T23:48:03.857 INFO:teuthology.orchestra.run.smithi102.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-15T23:48:03.872 INFO:teuthology.orchestra.run.smithi081.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 381 kB/s | 32 kB 00:00 2024-09-15T23:48:03.924 INFO:teuthology.orchestra.run.smithi102.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 740 kB/s | 49 kB 00:00 2024-09-15T23:48:03.981 INFO:teuthology.orchestra.run.smithi081.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-15T23:48:04.016 INFO:teuthology.orchestra.run.smithi102.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 179 kB/s | 16 kB 00:00 2024-09-15T23:48:04.019 INFO:teuthology.orchestra.run.smithi031.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.1 MB/s | 2.4 MB 00:01 2024-09-15T23:48:04.056 INFO:teuthology.orchestra.run.smithi081.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-15T23:48:04.100 INFO:teuthology.orchestra.run.smithi102.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.3 MB/s | 285 kB 00:00 2024-09-15T23:48:04.140 INFO:teuthology.orchestra.run.smithi081.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.5 MB/s | 127 kB 00:00 2024-09-15T23:48:04.240 INFO:teuthology.orchestra.run.smithi081.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 4.9 MB/s | 506 kB 00:00 2024-09-15T23:48:04.258 INFO:teuthology.orchestra.run.smithi102.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 4.7 MB/s | 765 kB 00:00 2024-09-15T23:48:04.351 INFO:teuthology.orchestra.run.smithi102.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 7.3 MB/s | 7.4 MB 00:01 2024-09-15T23:48:04.376 INFO:teuthology.orchestra.run.smithi102.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 156 kB/s | 18 kB 00:00 2024-09-15T23:48:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-15T23:48:04.535 INFO:teuthology.orchestra.run.smithi102.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T23:48:04.568 INFO:teuthology.orchestra.run.smithi102.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-15T23:48:04.594 INFO:teuthology.orchestra.run.smithi102.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-15T23:48:04.616 INFO:teuthology.orchestra.run.smithi081.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-15T23:48:04.620 INFO:teuthology.orchestra.run.smithi031.stdout:(5/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-09-15T23:48:04.644 INFO:teuthology.orchestra.run.smithi102.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-15T23:48:04.666 INFO:teuthology.orchestra.run.smithi081.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 905 kB/s | 45 kB 00:00 2024-09-15T23:48:04.669 INFO:teuthology.orchestra.run.smithi102.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-15T23:48:04.695 INFO:teuthology.orchestra.run.smithi102.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-15T23:48:04.720 INFO:teuthology.orchestra.run.smithi102.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T23:48:04.737 INFO:teuthology.orchestra.run.smithi031.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 2.1 MB/s | 1.5 MB 00:00 2024-09-15T23:48:04.741 INFO:teuthology.orchestra.run.smithi081.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.8 MB/s | 136 kB 00:00 2024-09-15T23:48:04.745 INFO:teuthology.orchestra.run.smithi102.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-15T23:48:04.808 INFO:teuthology.orchestra.run.smithi081.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-15T23:48:04.945 INFO:teuthology.orchestra.run.smithi102.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-15T23:48:05.005 INFO:teuthology.orchestra.run.smithi102.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-15T23:48:05.025 INFO:teuthology.orchestra.run.smithi081.stdout:(22/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 7.0 MB/s | 17 MB 00:02 2024-09-15T23:48:05.029 INFO:teuthology.orchestra.run.smithi102.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-15T23:48:05.051 INFO:teuthology.orchestra.run.smithi081.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 1.3 MB/s | 320 kB 00:00 2024-09-15T23:48:05.055 INFO:teuthology.orchestra.run.smithi102.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 430 kB/s | 22 kB 00:00 2024-09-15T23:48:05.084 INFO:teuthology.orchestra.run.smithi081.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-15T23:48:05.109 INFO:teuthology.orchestra.run.smithi081.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T23:48:05.135 INFO:teuthology.orchestra.run.smithi081.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-15T23:48:05.193 INFO:teuthology.orchestra.run.smithi081.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-15T23:48:05.286 INFO:teuthology.orchestra.run.smithi081.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 3.1 MB 00:00 2024-09-15T23:48:05.311 INFO:teuthology.orchestra.run.smithi081.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 272 kB/s | 32 kB 00:00 2024-09-15T23:48:05.336 INFO:teuthology.orchestra.run.smithi081.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-15T23:48:05.480 INFO:teuthology.orchestra.run.smithi102.stdout:(52/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 12 MB/s | 49 MB 00:04 2024-09-15T23:48:05.505 INFO:teuthology.orchestra.run.smithi102.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 73 kB/s | 33 kB 00:00 2024-09-15T23:48:05.547 INFO:teuthology.orchestra.run.smithi102.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 606 kB/s | 313 kB 00:00 2024-09-15T23:48:05.573 INFO:teuthology.orchestra.run.smithi102.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-15T23:48:05.598 INFO:teuthology.orchestra.run.smithi102.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T23:48:05.624 INFO:teuthology.orchestra.run.smithi102.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-15T23:48:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-15T23:48:05.708 INFO:teuthology.orchestra.run.smithi102.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-15T23:48:05.766 INFO:teuthology.orchestra.run.smithi102.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 421 kB/s | 24 kB 00:00 2024-09-15T23:48:05.796 INFO:teuthology.orchestra.run.smithi081.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-15T23:48:05.799 INFO:teuthology.orchestra.run.smithi102.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-15T23:48:05.812 INFO:teuthology.orchestra.run.smithi031.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 3.9 MB/s | 4.7 MB 00:01 2024-09-15T23:48:05.833 INFO:teuthology.orchestra.run.smithi102.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-15T23:48:05.863 INFO:teuthology.orchestra.run.smithi081.stdout:(32/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.5 MB/s | 3.6 MB 00:00 2024-09-15T23:48:05.879 INFO:teuthology.orchestra.run.smithi102.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-15T23:48:05.887 INFO:teuthology.orchestra.run.smithi081.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-15T23:48:05.892 INFO:teuthology.orchestra.run.smithi102.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-15T23:48:05.913 INFO:teuthology.orchestra.run.smithi081.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 980 kB/s | 49 kB 00:00 2024-09-15T23:48:05.917 INFO:teuthology.orchestra.run.smithi102.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-15T23:48:05.938 INFO:teuthology.orchestra.run.smithi081.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 327 kB/s | 16 kB 00:00 2024-09-15T23:48:05.963 INFO:teuthology.orchestra.run.smithi081.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-15T23:48:06.001 INFO:teuthology.orchestra.run.smithi102.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-09-15T23:48:06.005 INFO:teuthology.orchestra.run.smithi081.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-15T23:48:06.026 INFO:teuthology.orchestra.run.smithi081.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-15T23:48:06.101 INFO:teuthology.orchestra.run.smithi102.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T23:48:06.185 INFO:teuthology.orchestra.run.smithi102.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-15T23:48:06.219 INFO:teuthology.orchestra.run.smithi102.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-15T23:48:06.244 INFO:teuthology.orchestra.run.smithi102.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-15T23:48:06.311 INFO:teuthology.orchestra.run.smithi102.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-15T23:48:06.353 INFO:teuthology.orchestra.run.smithi102.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-15T23:48:06.378 INFO:teuthology.orchestra.run.smithi102.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-15T23:48:06.428 INFO:teuthology.orchestra.run.smithi102.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-15T23:48:06.481 INFO:teuthology.orchestra.run.smithi081.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 14 MB/s | 49 MB 00:03 2024-09-15T23:48:06.504 INFO:teuthology.orchestra.run.smithi102.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-15T23:48:06.512 INFO:teuthology.orchestra.run.smithi031.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 4.6 MB/s | 21 MB 00:04 2024-09-15T23:48:06.537 INFO:teuthology.orchestra.run.smithi102.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-15T23:48:06.553 INFO:teuthology.orchestra.run.smithi031.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 612 kB/s | 25 kB 00:00 2024-09-15T23:48:06.571 INFO:teuthology.orchestra.run.smithi102.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-15T23:48:06.596 INFO:teuthology.orchestra.run.smithi102.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-15T23:48:06.597 INFO:teuthology.orchestra.run.smithi081.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 135 kB/s | 80 kB 00:00 2024-09-15T23:48:06.621 INFO:teuthology.orchestra.run.smithi102.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-15T23:48:06.647 INFO:teuthology.orchestra.run.smithi102.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-15T23:48:06.723 INFO:teuthology.orchestra.run.smithi081.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-15T23:48:06.747 INFO:teuthology.orchestra.run.smithi102.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-15T23:48:06.757 INFO:teuthology.orchestra.run.smithi081.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-15T23:48:06.772 INFO:teuthology.orchestra.run.smithi102.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 230 kB/s | 35 kB 00:00 2024-09-15T23:48:06.781 INFO:teuthology.orchestra.run.smithi081.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 636 kB/s | 191 kB 00:00 2024-09-15T23:48:06.806 INFO:teuthology.orchestra.run.smithi102.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-15T23:48:06.807 INFO:teuthology.orchestra.run.smithi081.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T23:48:06.831 INFO:teuthology.orchestra.run.smithi102.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-15T23:48:06.857 INFO:teuthology.orchestra.run.smithi102.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T23:48:06.857 INFO:teuthology.orchestra.run.smithi081.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T23:48:06.882 INFO:teuthology.orchestra.run.smithi102.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-15T23:48:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-15T23:48:06.916 INFO:teuthology.orchestra.run.smithi081.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-09-15T23:48:06.942 INFO:teuthology.orchestra.run.smithi081.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-15T23:48:06.949 INFO:teuthology.orchestra.run.smithi102.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-15T23:48:06.974 INFO:teuthology.orchestra.run.smithi102.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-15T23:48:06.999 INFO:teuthology.orchestra.run.smithi102.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-15T23:48:07.025 INFO:teuthology.orchestra.run.smithi102.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-15T23:48:07.058 INFO:teuthology.orchestra.run.smithi102.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-15T23:48:07.083 INFO:teuthology.orchestra.run.smithi102.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 637 kB/s | 54 kB 00:00 2024-09-15T23:48:07.117 INFO:teuthology.orchestra.run.smithi102.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 709 kB/s | 42 kB 00:00 2024-09-15T23:48:07.125 INFO:teuthology.orchestra.run.smithi081.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-09-15T23:48:07.143 INFO:teuthology.orchestra.run.smithi102.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-09-15T23:48:07.176 INFO:teuthology.orchestra.run.smithi102.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T23:48:07.184 INFO:teuthology.orchestra.run.smithi081.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-09-15T23:48:07.209 INFO:teuthology.orchestra.run.smithi081.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-15T23:48:07.210 INFO:teuthology.orchestra.run.smithi102.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-09-15T23:48:07.235 INFO:teuthology.orchestra.run.smithi102.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-15T23:48:07.268 INFO:teuthology.orchestra.run.smithi081.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-15T23:48:07.277 INFO:teuthology.orchestra.run.smithi102.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-15T23:48:07.293 INFO:teuthology.orchestra.run.smithi081.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-15T23:48:07.302 INFO:teuthology.orchestra.run.smithi102.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-09-15T23:48:07.318 INFO:teuthology.orchestra.run.smithi081.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-15T23:48:07.320 INFO:teuthology.orchestra.run.smithi031.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 7.0 MB/s | 11 MB 00:01 2024-09-15T23:48:07.336 INFO:teuthology.orchestra.run.smithi102.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-15T23:48:07.343 INFO:teuthology.orchestra.run.smithi081.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-15T23:48:07.361 INFO:teuthology.orchestra.run.smithi102.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-15T23:48:07.369 INFO:teuthology.orchestra.run.smithi081.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-15T23:48:07.386 INFO:teuthology.orchestra.run.smithi102.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-09-15T23:48:07.396 INFO:teuthology.orchestra.run.smithi031.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.0 MB/s | 228 kB 00:00 2024-09-15T23:48:07.402 INFO:teuthology.orchestra.run.smithi081.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-15T23:48:07.411 INFO:teuthology.orchestra.run.smithi102.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-15T23:48:07.427 INFO:teuthology.orchestra.run.smithi081.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-15T23:48:07.471 INFO:teuthology.orchestra.run.smithi031.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 361 kB/s | 27 kB 00:00 2024-09-15T23:48:07.528 INFO:teuthology.orchestra.run.smithi081.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-15T23:48:07.546 INFO:teuthology.orchestra.run.smithi031.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 422 kB/s | 32 kB 00:00 2024-09-15T23:48:07.570 INFO:teuthology.orchestra.run.smithi081.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-15T23:48:07.571 INFO:teuthology.orchestra.run.smithi102.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-15T23:48:07.595 INFO:teuthology.orchestra.run.smithi081.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-15T23:48:07.621 INFO:teuthology.orchestra.run.smithi081.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-15T23:48:07.629 INFO:teuthology.orchestra.run.smithi102.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-15T23:48:07.671 INFO:teuthology.orchestra.run.smithi102.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-15T23:48:07.687 INFO:teuthology.orchestra.run.smithi081.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-15T23:48:07.696 INFO:teuthology.orchestra.run.smithi102.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.5 MB/s | 308 kB 00:00 2024-09-15T23:48:07.713 INFO:teuthology.orchestra.run.smithi102.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 237 kB/s | 19 kB 00:00 2024-09-15T23:48:07.730 INFO:teuthology.orchestra.run.smithi102.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 773 kB/s | 26 kB 00:00 2024-09-15T23:48:07.738 INFO:teuthology.orchestra.run.smithi081.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-15T23:48:07.746 INFO:teuthology.orchestra.run.smithi031.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 3.7 MB/s | 751 kB 00:00 2024-09-15T23:48:07.747 INFO:teuthology.orchestra.run.smithi102.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-15T23:48:07.763 INFO:teuthology.orchestra.run.smithi081.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-15T23:48:07.772 INFO:teuthology.orchestra.run.smithi102.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-15T23:48:07.798 INFO:teuthology.orchestra.run.smithi102.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-15T23:48:07.856 INFO:teuthology.orchestra.run.smithi102.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-15T23:48:07.881 INFO:teuthology.orchestra.run.smithi102.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-15T23:48:07.907 INFO:teuthology.orchestra.run.smithi102.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-09-15T23:48:07.914 INFO:teuthology.orchestra.run.smithi031.stdout:(15/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 5.3 MB/s | 17 MB 00:03 2024-09-15T23:48:07.932 INFO:teuthology.orchestra.run.smithi102.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-15T23:48:07.939 INFO:teuthology.orchestra.run.smithi031.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 851 kB/s | 163 kB 00:00 2024-09-15T23:48:07.957 INFO:teuthology.orchestra.run.smithi102.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-15T23:48:07.997 INFO:teuthology.orchestra.run.smithi031.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.5 MB/s | 127 kB 00:00 2024-09-15T23:48:07.999 INFO:teuthology.orchestra.run.smithi102.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-15T23:48:08.041 INFO:teuthology.orchestra.run.smithi102.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-15T23:48:08.066 INFO:teuthology.orchestra.run.smithi102.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-15T23:48:08.100 INFO:teuthology.orchestra.run.smithi102.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.6 MB/s | 220 kB 00:00 2024-09-15T23:48:08.148 INFO:teuthology.orchestra.run.smithi031.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 2.4 MB/s | 506 kB 00:00 2024-09-15T23:48:08.167 INFO:teuthology.orchestra.run.smithi102.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-15T23:48:08.193 INFO:teuthology.orchestra.run.smithi102.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-15T23:48:08.213 INFO:teuthology.orchestra.run.smithi081.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-15T23:48:08.218 INFO:teuthology.orchestra.run.smithi102.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-15T23:48:08.231 INFO:teuthology.orchestra.run.smithi031.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 540 kB/s | 45 kB 00:00 2024-09-15T23:48:08.243 INFO:teuthology.orchestra.run.smithi102.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-15T23:48:08.269 INFO:teuthology.orchestra.run.smithi102.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-15T23:48:08.282 INFO:teuthology.orchestra.run.smithi031.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-15T23:48:08.293 INFO:teuthology.orchestra.run.smithi102.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-15T23:48:08.319 INFO:teuthology.orchestra.run.smithi102.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-15T23:48:08.332 INFO:teuthology.orchestra.run.smithi031.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.2 MB/s | 161 kB 00:00 2024-09-15T23:48:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T23:48:08.378 INFO:teuthology.orchestra.run.smithi102.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T23:48:08.403 INFO:teuthology.orchestra.run.smithi102.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T23:48:08.415 INFO:teuthology.orchestra.run.smithi031.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T23:48:08.437 INFO:teuthology.orchestra.run.smithi102.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-15T23:48:08.479 INFO:teuthology.orchestra.run.smithi102.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-09-15T23:48:08.504 INFO:teuthology.orchestra.run.smithi102.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-09-15T23:48:08.507 INFO:teuthology.orchestra.run.smithi031.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.2 MB/s | 300 kB 00:00 2024-09-15T23:48:08.529 INFO:teuthology.orchestra.run.smithi102.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-15T23:48:08.554 INFO:teuthology.orchestra.run.smithi102.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-15T23:48:08.574 INFO:teuthology.orchestra.run.smithi031.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 99 kB 00:00 2024-09-15T23:48:08.580 INFO:teuthology.orchestra.run.smithi102.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-15T23:48:08.605 INFO:teuthology.orchestra.run.smithi102.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-15T23:48:08.630 INFO:teuthology.orchestra.run.smithi102.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-15T23:48:08.650 INFO:teuthology.orchestra.run.smithi031.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T23:48:08.655 INFO:teuthology.orchestra.run.smithi102.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-15T23:48:08.681 INFO:teuthology.orchestra.run.smithi102.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-15T23:48:08.706 INFO:teuthology.orchestra.run.smithi102.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 984 kB/s | 74 kB 00:00 2024-09-15T23:48:08.731 INFO:teuthology.orchestra.run.smithi102.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-15T23:48:08.757 INFO:teuthology.orchestra.run.smithi102.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T23:48:08.782 INFO:teuthology.orchestra.run.smithi102.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-15T23:48:08.807 INFO:teuthology.orchestra.run.smithi102.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T23:48:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-15T23:48:08.899 INFO:teuthology.orchestra.run.smithi102.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T23:48:09.072 INFO:teuthology.orchestra.run.smithi081.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-09-15T23:48:09.075 INFO:teuthology.orchestra.run.smithi102.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-15T23:48:09.142 INFO:teuthology.orchestra.run.smithi102.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-15T23:48:09.184 INFO:teuthology.orchestra.run.smithi102.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 982 kB/s | 41 kB 00:00 2024-09-15T23:48:09.226 INFO:teuthology.orchestra.run.smithi102.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-15T23:48:09.284 INFO:teuthology.orchestra.run.smithi102.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-15T23:48:09.360 INFO:teuthology.orchestra.run.smithi102.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-15T23:48:09.425 INFO:teuthology.orchestra.run.smithi031.stdout:(26/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 4.0 MB/s | 3.1 MB 00:00 2024-09-15T23:48:09.502 INFO:teuthology.orchestra.run.smithi102.stdout:(155/156): librados2-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.4 MB 00:00 2024-09-15T23:48:09.525 INFO:teuthology.orchestra.run.smithi031.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-15T23:48:09.611 INFO:teuthology.orchestra.run.smithi102.stdout:(156/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 4.1 MB/s | 3.2 MB 00:00 2024-09-15T23:48:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:48:09.617 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-15T23:48:09.626 INFO:teuthology.orchestra.run.smithi031.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 319 kB/s | 32 kB 00:00 2024-09-15T23:48:09.656 INFO:teuthology.orchestra.run.smithi081.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.5 MB/s | 910 kB 00:00 2024-09-15T23:48:09.692 INFO:teuthology.orchestra.run.smithi031.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.1 MB/s | 146 kB 00:00 2024-09-15T23:48:09.781 INFO:teuthology.orchestra.run.smithi081.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-15T23:48:09.809 INFO:teuthology.orchestra.run.smithi031.stdout:(30/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 3.0 MB/s | 5.3 MB 00:01 2024-09-15T23:48:09.848 INFO:teuthology.orchestra.run.smithi081.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.3 MB/s | 4.9 MB 00:02 2024-09-15T23:48:09.948 INFO:teuthology.orchestra.run.smithi081.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-15T23:48:09.982 INFO:teuthology.orchestra.run.smithi081.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-15T23:48:10.032 INFO:teuthology.orchestra.run.smithi081.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-15T23:48:10.082 INFO:teuthology.orchestra.run.smithi081.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-15T23:48:10.224 INFO:teuthology.orchestra.run.smithi081.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-15T23:48:10.366 INFO:teuthology.orchestra.run.smithi081.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-15T23:48:10.416 INFO:teuthology.orchestra.run.smithi081.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-15T23:48:10.460 INFO:teuthology.orchestra.run.smithi031.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 4.6 MB/s | 3.6 MB 00:00 2024-09-15T23:48:10.466 INFO:teuthology.orchestra.run.smithi081.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-15T23:48:10.584 INFO:teuthology.orchestra.run.smithi081.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-15T23:48:10.635 INFO:teuthology.orchestra.run.smithi031.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 1.4 MB/s | 249 kB 00:00 2024-09-15T23:48:10.650 INFO:teuthology.orchestra.run.smithi081.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 523 kB/s | 35 kB 00:00 2024-09-15T23:48:10.702 INFO:teuthology.orchestra.run.smithi031.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 741 kB/s | 49 kB 00:00 2024-09-15T23:48:10.734 INFO:teuthology.orchestra.run.smithi081.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.9 MB/s | 6.0 MB 00:00 2024-09-15T23:48:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 246 kB/s | 16 kB 00:00 2024-09-15T23:48:10.793 INFO:teuthology.orchestra.run.smithi081.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-15T23:48:10.801 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-15T23:48:10.852 INFO:teuthology.orchestra.run.smithi031.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.3 MB/s | 285 kB 00:00 2024-09-15T23:48:10.868 INFO:teuthology.orchestra.run.smithi081.stdout:(83/156): postgresql-server-13.16-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-09-15T23:48:10.893 INFO:teuthology.orchestra.run.smithi081.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-15T23:48:10.935 INFO:teuthology.orchestra.run.smithi081.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-15T23:48:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-15T23:48:10.951 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-15T23:48:10.963 INFO:teuthology.orchestra.run.smithi081.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-09-15T23:48:10.994 INFO:teuthology.orchestra.run.smithi081.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-15T23:48:11.029 INFO:teuthology.orchestra.run.smithi081.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-15T23:48:11.036 INFO:teuthology.orchestra.run.smithi031.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-15T23:48:11.053 INFO:teuthology.orchestra.run.smithi081.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-15T23:48:11.096 INFO:teuthology.orchestra.run.smithi081.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-09-15T23:48:11.114 INFO:teuthology.orchestra.run.smithi081.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-09-15T23:48:11.139 INFO:teuthology.orchestra.run.smithi031.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 183 kB/s | 18 kB 00:00 2024-09-15T23:48:11.180 INFO:teuthology.orchestra.run.smithi081.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.6 MB/s | 5.1 MB 00:00 2024-09-15T23:48:11.196 INFO:teuthology.orchestra.run.smithi081.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-15T23:48:11.222 INFO:teuthology.orchestra.run.smithi081.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-15T23:48:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-15T23:48:11.288 INFO:teuthology.orchestra.run.smithi081.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-15T23:48:11.314 INFO:teuthology.orchestra.run.smithi081.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-09-15T23:48:11.356 INFO:teuthology.orchestra.run.smithi081.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-09-15T23:48:11.381 INFO:teuthology.orchestra.run.smithi081.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-15T23:48:11.423 INFO:teuthology.orchestra.run.smithi081.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-09-15T23:48:11.428 INFO:teuthology.orchestra.run.smithi031.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 5.2 MB/s | 801 kB 00:00 2024-09-15T23:48:11.448 INFO:teuthology.orchestra.run.smithi081.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-15T23:48:11.512 INFO:teuthology.orchestra.run.smithi031.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-15T23:48:11.596 INFO:teuthology.orchestra.run.smithi031.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-15T23:48:11.607 INFO:teuthology.orchestra.run.smithi081.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-15T23:48:11.632 INFO:teuthology.orchestra.run.smithi081.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 244 kB/s | 45 kB 00:00 2024-09-15T23:48:11.657 INFO:teuthology.orchestra.run.smithi081.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 272 kB/s | 64 kB 00:00 2024-09-15T23:48:11.696 INFO:teuthology.orchestra.run.smithi031.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-15T23:48:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-15T23:48:11.733 INFO:teuthology.orchestra.run.smithi081.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-15T23:48:11.758 INFO:teuthology.orchestra.run.smithi081.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-15T23:48:11.763 INFO:teuthology.orchestra.run.smithi031.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-15T23:48:11.838 INFO:teuthology.orchestra.run.smithi031.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-15T23:48:11.875 INFO:teuthology.orchestra.run.smithi081.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-15T23:48:11.900 INFO:teuthology.orchestra.run.smithi081.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T23:48:11.922 INFO:teuthology.orchestra.run.smithi031.stdout:(45/156): ceph-mgr-diskprediction-local-19.1.1- 3.5 MB/s | 7.4 MB 00:02 2024-09-15T23:48:11.926 INFO:teuthology.orchestra.run.smithi081.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T23:48:11.947 INFO:teuthology.orchestra.run.smithi031.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-15T23:48:11.951 INFO:teuthology.orchestra.run.smithi081.stdout:(110/156): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-09-15T23:48:11.985 INFO:teuthology.orchestra.run.smithi081.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-15T23:48:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-15T23:48:12.052 INFO:teuthology.orchestra.run.smithi081.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-15T23:48:12.064 INFO:teuthology.orchestra.run.smithi031.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-15T23:48:12.077 INFO:teuthology.orchestra.run.smithi081.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T23:48:12.103 INFO:teuthology.orchestra.run.smithi081.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T23:48:12.128 INFO:teuthology.orchestra.run.smithi081.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-15T23:48:12.131 INFO:teuthology.orchestra.run.smithi031.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-15T23:48:12.170 INFO:teuthology.orchestra.run.smithi081.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-09-15T23:48:12.173 INFO:teuthology.orchestra.run.smithi031.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-15T23:48:12.195 INFO:teuthology.orchestra.run.smithi081.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-15T23:48:12.207 INFO:teuthology.orchestra.run.smithi031.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 652 kB/s | 22 kB 00:00 2024-09-15T23:48:12.229 INFO:teuthology.orchestra.run.smithi081.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-15T23:48:12.240 INFO:teuthology.orchestra.run.smithi031.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-09-15T23:48:12.254 INFO:teuthology.orchestra.run.smithi081.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-15T23:48:12.265 INFO:teuthology.orchestra.run.smithi031.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-15T23:48:12.279 INFO:teuthology.orchestra.run.smithi081.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-15T23:48:12.291 INFO:teuthology.orchestra.run.smithi031.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-15T23:48:12.305 INFO:teuthology.orchestra.run.smithi081.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-15T23:48:12.330 INFO:teuthology.orchestra.run.smithi081.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-15T23:48:12.333 INFO:teuthology.orchestra.run.smithi031.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-15T23:48:12.355 INFO:teuthology.orchestra.run.smithi081.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-15T23:48:12.380 INFO:teuthology.orchestra.run.smithi081.stdout:(125/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-15T23:48:12.391 INFO:teuthology.orchestra.run.smithi031.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-15T23:48:12.406 INFO:teuthology.orchestra.run.smithi081.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-15T23:48:12.433 INFO:teuthology.orchestra.run.smithi031.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-15T23:48:12.439 INFO:teuthology.orchestra.run.smithi081.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 766 kB/s | 20 kB 00:00 2024-09-15T23:48:12.458 INFO:teuthology.orchestra.run.smithi031.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-15T23:48:12.465 INFO:teuthology.orchestra.run.smithi081.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-09-15T23:48:12.484 INFO:teuthology.orchestra.run.smithi031.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-15T23:48:12.515 INFO:teuthology.orchestra.run.smithi081.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-15T23:48:12.517 INFO:teuthology.orchestra.run.smithi031.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-15T23:48:12.532 INFO:teuthology.orchestra.run.smithi081.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-09-15T23:48:12.557 INFO:teuthology.orchestra.run.smithi081.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-15T23:48:12.559 INFO:teuthology.orchestra.run.smithi031.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-15T23:48:12.574 INFO:teuthology.orchestra.run.smithi081.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-15T23:48:12.591 INFO:teuthology.orchestra.run.smithi081.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-15T23:48:12.625 INFO:teuthology.orchestra.run.smithi081.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-09-15T23:48:12.626 INFO:teuthology.orchestra.run.smithi031.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-15T23:48:12.650 INFO:teuthology.orchestra.run.smithi081.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-15T23:48:12.652 INFO:teuthology.orchestra.run.smithi031.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-09-15T23:48:12.675 INFO:teuthology.orchestra.run.smithi081.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-15T23:48:12.685 INFO:teuthology.orchestra.run.smithi031.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-15T23:48:12.700 INFO:teuthology.orchestra.run.smithi081.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-15T23:48:12.710 INFO:teuthology.orchestra.run.smithi031.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-15T23:48:12.725 INFO:teuthology.orchestra.run.smithi081.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-09-15T23:48:12.736 INFO:teuthology.orchestra.run.smithi031.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 771 kB/s | 38 kB 00:00 2024-09-15T23:48:12.751 INFO:teuthology.orchestra.run.smithi081.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-15T23:48:12.776 INFO:teuthology.orchestra.run.smithi081.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-15T23:48:12.801 INFO:teuthology.orchestra.run.smithi081.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-15T23:48:12.827 INFO:teuthology.orchestra.run.smithi081.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-15T23:48:12.853 INFO:teuthology.orchestra.run.smithi081.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-15T23:48:12.861 INFO:teuthology.orchestra.run.smithi031.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-15T23:48:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T23:48:12.913 INFO:teuthology.orchestra.run.smithi081.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 339 kB/s | 20 kB 00:00 2024-09-15T23:48:12.928 INFO:teuthology.orchestra.run.smithi031.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-15T23:48:12.938 INFO:teuthology.orchestra.run.smithi081.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T23:48:12.963 INFO:teuthology.orchestra.run.smithi081.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-15T23:48:12.970 INFO:teuthology.orchestra.run.smithi031.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-15T23:48:13.003 INFO:teuthology.orchestra.run.smithi031.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-15T23:48:13.037 INFO:teuthology.orchestra.run.smithi031.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-15T23:48:13.055 INFO:teuthology.orchestra.run.smithi081.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T23:48:13.096 INFO:teuthology.orchestra.run.smithi031.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-15T23:48:13.171 INFO:teuthology.orchestra.run.smithi031.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-15T23:48:13.179 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-15T23:48:13.180 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-15T23:48:13.196 INFO:teuthology.orchestra.run.smithi031.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-09-15T23:48:13.238 INFO:teuthology.orchestra.run.smithi031.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-15T23:48:13.272 INFO:teuthology.orchestra.run.smithi081.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-15T23:48:13.339 INFO:teuthology.orchestra.run.smithi031.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-09-15T23:48:13.364 INFO:teuthology.orchestra.run.smithi031.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-15T23:48:13.398 INFO:teuthology.orchestra.run.smithi031.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-15T23:48:13.423 INFO:teuthology.orchestra.run.smithi031.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 435 kB/s | 26 kB 00:00 2024-09-15T23:48:13.448 INFO:teuthology.orchestra.run.smithi031.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-15T23:48:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-15T23:48:13.515 INFO:teuthology.orchestra.run.smithi031.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-09-15T23:48:13.531 INFO:teuthology.orchestra.run.smithi081.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 7.0 MB/s | 1.8 MB 00:00 2024-09-15T23:48:13.558 INFO:teuthology.orchestra.run.smithi031.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-15T23:48:13.581 INFO:teuthology.orchestra.run.smithi081.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 820 kB/s | 41 kB 00:00 2024-09-15T23:48:13.633 INFO:teuthology.orchestra.run.smithi031.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-15T23:48:13.640 INFO:teuthology.orchestra.run.smithi081.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-15T23:48:13.658 INFO:teuthology.orchestra.run.smithi031.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-15T23:48:13.684 INFO:teuthology.orchestra.run.smithi031.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T23:48:13.701 INFO:teuthology.orchestra.run.smithi031.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-15T23:48:13.715 INFO:teuthology.orchestra.run.smithi081.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-15T23:48:13.726 INFO:teuthology.orchestra.run.smithi031.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-15T23:48:13.751 INFO:teuthology.orchestra.run.smithi031.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-15T23:48:13.776 INFO:teuthology.orchestra.run.smithi031.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-15T23:48:13.790 INFO:teuthology.orchestra.run.smithi081.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 3.2 MB/s | 2.3 MB 00:00 2024-09-15T23:48:13.801 INFO:teuthology.orchestra.run.smithi031.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-15T23:48:13.826 INFO:teuthology.orchestra.run.smithi031.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-15T23:48:13.840 INFO:teuthology.orchestra.run.smithi081.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-15T23:48:13.869 INFO:teuthology.orchestra.run.smithi031.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-15T23:48:13.911 INFO:teuthology.orchestra.run.smithi031.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-15T23:48:13.953 INFO:teuthology.orchestra.run.smithi031.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-15T23:48:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 866 kB/s | 36 kB 00:00 2024-09-15T23:48:14.178 INFO:teuthology.orchestra.run.smithi031.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-15T23:48:14.203 INFO:teuthology.orchestra.run.smithi031.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 227 kB/s | 47 kB 00:00 2024-09-15T23:48:14.207 INFO:teuthology.orchestra.run.smithi081.stdout:(156/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 2.5 MB/s | 3.2 MB 00:01 2024-09-15T23:48:14.214 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:48:14.214 INFO:teuthology.orchestra.run.smithi081.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-15T23:48:14.228 INFO:teuthology.orchestra.run.smithi031.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-15T23:48:14.254 INFO:teuthology.orchestra.run.smithi031.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T23:48:14.279 INFO:teuthology.orchestra.run.smithi031.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T23:48:14.304 INFO:teuthology.orchestra.run.smithi031.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T23:48:14.330 INFO:teuthology.orchestra.run.smithi031.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-15T23:48:14.871 INFO:teuthology.orchestra.run.smithi031.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-15T23:48:14.922 INFO:teuthology.orchestra.run.smithi031.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-15T23:48:14.955 INFO:teuthology.orchestra.run.smithi031.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-15T23:48:14.981 INFO:teuthology.orchestra.run.smithi031.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-15T23:48:15.098 INFO:teuthology.orchestra.run.smithi031.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-15T23:48:15.123 INFO:teuthology.orchestra.run.smithi031.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T23:48:15.148 INFO:teuthology.orchestra.run.smithi031.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T23:48:15.174 INFO:teuthology.orchestra.run.smithi031.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T23:48:15.448 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-15T23:48:15.599 INFO:teuthology.orchestra.run.smithi031.stdout:(111/156): ceph-test-19.1.1-326.g8921c8ef.el9.x 5.4 MB/s | 49 MB 00:09 2024-09-15T23:48:15.600 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-15T23:48:15.600 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-15T23:48:15.649 INFO:teuthology.orchestra.run.smithi031.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 654 kB/s | 863 kB 00:01 2024-09-15T23:48:15.675 INFO:teuthology.orchestra.run.smithi031.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 302 kB/s | 151 kB 00:00 2024-09-15T23:48:15.708 INFO:teuthology.orchestra.run.smithi031.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T23:48:15.733 INFO:teuthology.orchestra.run.smithi031.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-15T23:48:15.758 INFO:teuthology.orchestra.run.smithi031.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 574 kB/s | 14 kB 00:00 2024-09-15T23:48:15.784 INFO:teuthology.orchestra.run.smithi031.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-15T23:48:15.826 INFO:teuthology.orchestra.run.smithi031.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-15T23:48:15.859 INFO:teuthology.orchestra.run.smithi031.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-15T23:48:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-09-15T23:48:15.918 INFO:teuthology.orchestra.run.smithi031.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-15T23:48:15.952 INFO:teuthology.orchestra.run.smithi031.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-15T23:48:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-15T23:48:15.994 INFO:teuthology.orchestra.run.smithi031.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-09-15T23:48:16.011 INFO:teuthology.orchestra.run.smithi031.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2024-09-15T23:48:16.028 INFO:teuthology.orchestra.run.smithi031.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-09-15T23:48:16.045 INFO:teuthology.orchestra.run.smithi031.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-15T23:48:16.062 INFO:teuthology.orchestra.run.smithi031.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 591 kB/s | 20 kB 00:00 2024-09-15T23:48:16.087 INFO:teuthology.orchestra.run.smithi031.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T23:48:16.113 INFO:teuthology.orchestra.run.smithi031.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T23:48:16.138 INFO:teuthology.orchestra.run.smithi031.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T23:48:16.180 INFO:teuthology.orchestra.run.smithi031.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-15T23:48:16.197 INFO:teuthology.orchestra.run.smithi031.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 991 kB/s | 16 kB 00:00 2024-09-15T23:48:16.222 INFO:teuthology.orchestra.run.smithi031.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-15T23:48:16.256 INFO:teuthology.orchestra.run.smithi031.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-15T23:48:16.273 INFO:teuthology.orchestra.run.smithi031.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-15T23:48:16.290 INFO:teuthology.orchestra.run.smithi031.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 913 kB/s | 31 kB 00:00 2024-09-15T23:48:16.315 INFO:teuthology.orchestra.run.smithi031.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T23:48:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-15T23:48:16.366 INFO:teuthology.orchestra.run.smithi031.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-15T23:48:16.391 INFO:teuthology.orchestra.run.smithi031.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-15T23:48:16.416 INFO:teuthology.orchestra.run.smithi031.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-15T23:48:16.450 INFO:teuthology.orchestra.run.smithi031.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-15T23:48:16.475 INFO:teuthology.orchestra.run.smithi031.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-15T23:48:16.517 INFO:teuthology.orchestra.run.smithi031.stdout:(145/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-15T23:48:16.550 INFO:teuthology.orchestra.run.smithi031.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-15T23:48:16.626 INFO:teuthology.orchestra.run.smithi031.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-09-15T23:48:16.676 INFO:teuthology.orchestra.run.smithi031.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-15T23:48:16.801 INFO:teuthology.orchestra.run.smithi031.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-09-15T23:48:16.910 INFO:teuthology.orchestra.run.smithi031.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-09-15T23:48:16.985 INFO:teuthology.orchestra.run.smithi031.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 543 kB/s | 41 kB 00:00 2024-09-15T23:48:17.061 INFO:teuthology.orchestra.run.smithi031.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-15T23:48:17.161 INFO:teuthology.orchestra.run.smithi031.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-15T23:48:17.261 INFO:teuthology.orchestra.run.smithi031.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-15T23:48:17.678 INFO:teuthology.orchestra.run.smithi031.stdout:(155/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 3.0 MB/s | 3.2 MB 00:01 2024-09-15T23:48:17.794 INFO:teuthology.orchestra.run.smithi031.stdout:(156/156): librados2-19.1.1-326.g8921c8ef.el9.x 2.8 MB/s | 3.4 MB 00:01 2024-09-15T23:48:17.802 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:48:17.802 INFO:teuthology.orchestra.run.smithi031.stdout:Total 13 MB/s | 226 MB 00:16 2024-09-15T23:48:17.862 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-15T23:48:17.862 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-15T23:48:19.084 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-15T23:48:19.255 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-15T23:48:19.256 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-15T23:48:19.557 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-15T23:48:19.657 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T23:48:19.707 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T23:48:19.767 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T23:48:19.799 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T23:48:19.825 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T23:48:19.883 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T23:48:20.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T23:48:20.318 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:20.439 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:20.460 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:20.543 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:20.573 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:48:20.742 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T23:48:20.804 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T23:48:20.856 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:20.961 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:20.982 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:21.053 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:21.075 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:21.094 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:21.293 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:21.474 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:21.582 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-15T23:48:21.582 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-15T23:48:21.606 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T23:48:21.718 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:21.791 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:22.017 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:22.123 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T23:48:22.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T23:48:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T23:48:22.289 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T23:48:22.354 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T23:48:22.398 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T23:48:22.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T23:48:22.474 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T23:48:22.507 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T23:48:22.534 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T23:48:22.603 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T23:48:22.657 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T23:48:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T23:48:22.731 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T23:48:22.807 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T23:48:22.850 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T23:48:22.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T23:48:22.922 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T23:48:22.991 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T23:48:23.036 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T23:48:23.110 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T23:48:23.252 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T23:48:23.305 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T23:48:23.826 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T23:48:23.909 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T23:48:23.939 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T23:48:23.981 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T23:48:24.019 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T23:48:24.061 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T23:48:24.241 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T23:48:24.305 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T23:48:24.310 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-15T23:48:24.360 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T23:48:24.432 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T23:48:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T23:48:24.465 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T23:48:24.491 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T23:48:24.501 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T23:48:24.547 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T23:48:24.561 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T23:48:24.584 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T23:48:24.603 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T23:48:24.627 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T23:48:24.629 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T23:48:24.669 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T23:48:24.680 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T23:48:24.803 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T23:48:24.845 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T23:48:24.886 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T23:48:24.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T23:48:25.012 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T23:48:25.021 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T23:48:25.068 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T23:48:25.122 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:25.153 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T23:48:25.245 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:25.353 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T23:48:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:25.435 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:48:25.595 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T23:48:25.606 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T23:48:25.672 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T23:48:25.702 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:25.723 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T23:48:25.759 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T23:48:25.803 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T23:48:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:25.837 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:25.842 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T23:48:25.881 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T23:48:25.900 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:25.912 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T23:48:25.929 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:25.943 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T23:48:25.952 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:25.977 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T23:48:26.010 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T23:48:26.052 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T23:48:26.086 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T23:48:26.149 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:26.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T23:48:26.242 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T23:48:26.274 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T23:48:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:26.375 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T23:48:26.424 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T23:48:26.444 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T23:48:26.576 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:26.646 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T23:48:27.096 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T23:48:27.127 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T23:48:27.133 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T23:48:27.169 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T23:48:27.179 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T23:48:27.213 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T23:48:27.234 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T23:48:27.278 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T23:48:27.310 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T23:48:27.310 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T23:48:27.354 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T23:48:27.360 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T23:48:27.387 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T23:48:27.404 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T23:48:27.414 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T23:48:27.482 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T23:48:27.492 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T23:48:27.533 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T23:48:27.545 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T23:48:27.576 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T23:48:27.620 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T23:48:27.695 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T23:48:27.717 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T23:48:27.731 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T23:48:27.747 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T23:48:27.775 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T23:48:27.803 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T23:48:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T23:48:27.871 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T23:48:27.917 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T23:48:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-15T23:48:27.998 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T23:48:28.040 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T23:48:28.138 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T23:48:28.177 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T23:48:28.194 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T23:48:28.335 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T23:48:28.382 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T23:48:28.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T23:48:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T23:48:28.486 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T23:48:28.605 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T23:48:28.721 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T23:48:28.806 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T23:48:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T23:48:28.836 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T23:48:28.879 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T23:48:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:28.917 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T23:48:28.959 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T23:48:29.033 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:48:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:29.123 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T23:48:29.151 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:48:29.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:48:29.203 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T23:48:29.265 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T23:48:29.331 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T23:48:29.363 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T23:48:29.379 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T23:48:29.399 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T23:48:29.436 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T23:48:29.445 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T23:48:29.462 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:29.482 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T23:48:29.524 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T23:48:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:48:29.566 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T23:48:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:29.643 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T23:48:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T23:48:29.708 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T23:48:29.752 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T23:48:29.793 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T23:48:29.847 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T23:48:29.894 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T23:48:29.902 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:29.927 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T23:48:29.928 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T23:48:29.955 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T23:48:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T23:48:29.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T23:48:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T23:48:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T23:48:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T23:48:30.145 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T23:48:30.179 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T23:48:30.270 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T23:48:30.298 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T23:48:30.300 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:30.335 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:30.372 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:30.426 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:30.456 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T23:48:30.511 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T23:48:30.529 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T23:48:30.623 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T23:48:30.624 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T23:48:30.669 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T23:48:30.706 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T23:48:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T23:48:30.746 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T23:48:30.775 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T23:48:30.818 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T23:48:30.836 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T23:48:30.860 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T23:48:30.895 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T23:48:30.902 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T23:48:30.936 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T23:48:30.937 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T23:48:30.991 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T23:48:31.012 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T23:48:31.025 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T23:48:31.029 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:31.064 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T23:48:31.074 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T23:48:31.101 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T23:48:31.105 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T23:48:31.123 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T23:48:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T23:48:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T23:48:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T23:48:31.208 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T23:48:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T23:48:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T23:48:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T23:48:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T23:48:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T23:48:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T23:48:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T23:48:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T23:48:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T23:48:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T23:48:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T23:48:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T23:48:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T23:48:31.975 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T23:48:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T23:48:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T23:48:32.055 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T23:48:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T23:48:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T23:48:32.254 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T23:48:32.334 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T23:48:32.366 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T23:48:32.549 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T23:48:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T23:48:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T23:48:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T23:48:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T23:48:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T23:48:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T23:48:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T23:48:32.818 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T23:48:32.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T23:48:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T23:48:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T23:48:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T23:48:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T23:48:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T23:48:33.297 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T23:48:33.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T23:48:33.348 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T23:48:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T23:48:33.414 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T23:48:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T23:48:33.459 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:33.503 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T23:48:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T23:48:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T23:48:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T23:48:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T23:48:34.071 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:34.106 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:34.112 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T23:48:34.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T23:48:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T23:48:34.195 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:34.195 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T23:48:34.195 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T23:48:34.195 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:34.241 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T23:48:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T23:48:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T23:48:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T23:48:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T23:48:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T23:48:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T23:48:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T23:48:34.640 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T23:48:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T23:48:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T23:48:34.728 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T23:48:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T23:48:34.764 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T23:48:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T23:48:34.798 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T23:48:34.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T23:48:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T23:48:34.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T23:48:35.078 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T23:48:35.105 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T23:48:35.129 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T23:48:35.152 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:35.242 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:35.274 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T23:48:35.320 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T23:48:35.797 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T23:48:35.822 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:35.892 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:35.895 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T23:48:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T23:48:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T23:48:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T23:48:36.096 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T23:48:36.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T23:48:36.193 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T23:48:36.292 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T23:48:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T23:48:36.521 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T23:48:36.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T23:48:36.656 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T23:48:37.350 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T23:48:37.470 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T23:48:38.244 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:38.279 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T23:48:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:38.879 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T23:48:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:38.916 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T23:48:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T23:48:38.972 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:38.972 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T23:48:38.972 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T23:48:38.972 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:39.008 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T23:48:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:39.436 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T23:48:39.466 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T23:48:39.503 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T23:48:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T23:48:40.039 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:40.110 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T23:48:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T23:48:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:43.135 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T23:48:43.173 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T23:48:43.275 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T23:48:43.275 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T23:48:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-15T23:48:45.885 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:46.064 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:46.080 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:46.080 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:46.080 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T23:48:46.080 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:47.567 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:47.593 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:47.593 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T23:48:47.593 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:47.593 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:47.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:47.840 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:48.268 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:48.290 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:48.290 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:48.290 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T23:48:48.290 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:48.290 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:48.291 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:48.406 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T23:48:48.454 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T23:48:48.509 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T23:48:48.693 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T23:48:48.743 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T23:48:48.812 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T23:48:48.900 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T23:48:49.040 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T23:48:49.110 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T23:48:49.150 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T23:48:49.178 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T23:48:49.643 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:49.669 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:49.692 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T23:48:49.747 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T23:48:49.775 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-09-15T23:48:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:50.671 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:50.686 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:50.687 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:50.687 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T23:48:50.687 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:50.820 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:48:50.852 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:48:51.815 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:48:51.844 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:48:51.947 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:48:52.113 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T23:48:52.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:48:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:48:52.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:52.184 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:52.184 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T23:48:52.185 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:52.185 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:52.208 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:48:52.217 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:48:52.314 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T23:48:52.380 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T23:48:52.408 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:52.431 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T23:48:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:52.432 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:52.867 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:52.890 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:52.891 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:52.891 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T23:48:52.891 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:52.891 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:52.891 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T23:48:53.061 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T23:48:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T23:48:53.299 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T23:48:53.307 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T23:48:53.333 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:48:53.351 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T23:48:53.356 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:48:53.356 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:53.356 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T23:48:53.356 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:48:53.356 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:48:53.357 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:53.400 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:48:53.422 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:48:53.422 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:53.422 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T23:48:53.422 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:53.427 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T23:48:53.467 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:48:53.490 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:53.516 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T23:48:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T23:48:53.711 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T23:48:53.749 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T23:48:53.777 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T23:48:53.793 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:48:53.815 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:48:53.815 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:53.815 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T23:48:53.816 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:48:53.816 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:48:53.816 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:48:54.164 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:54.193 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:54.224 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T23:48:54.294 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T23:48:54.324 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:48:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:48:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:48:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-15T23:48:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:55.824 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T23:48:55.842 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:55.842 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T23:48:55.842 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:56.366 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:48:56.394 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:48:56.496 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:48:56.657 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T23:48:56.687 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:48:56.712 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:56.771 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:48:56.783 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:48:56.879 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T23:48:56.943 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T23:48:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:57.412 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T23:48:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T23:48:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T23:48:57.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:57.654 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T23:48:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T23:48:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:57.685 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T23:48:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T23:48:57.899 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:48:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:57.967 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:48:57.989 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:48:57.990 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:57.990 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T23:48:57.990 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:58.042 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:48:58.066 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:48:58.066 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:58.066 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T23:48:58.067 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:48:58.067 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:48:58.067 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T23:48:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:48:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T23:48:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T23:48:58.355 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T23:48:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:48:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:48:58.391 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:48:58.392 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T23:48:58.392 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:48:58.392 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:48:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:48:58.396 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T23:48:58.415 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T23:48:58.436 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T23:48:58.469 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T23:48:58.496 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T23:48:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T23:48:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T23:48:58.694 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T23:48:58.743 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T23:48:58.774 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T23:48:58.803 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T23:48:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T23:48:58.849 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T23:48:58.985 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T23:48:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T23:48:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T23:48:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T23:48:59.433 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:48:59.472 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:48:59.493 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:48:59.545 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:59.591 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T23:48:59.629 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T23:48:59.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T23:48:59.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:49:00.321 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T23:49:00.410 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T23:49:00.455 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T23:49:00.556 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T23:49:00.588 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T23:49:00.613 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T23:49:00.639 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T23:49:00.640 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:00.732 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:00.732 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:00.824 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:00.829 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T23:49:00.850 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T23:49:00.854 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:00.855 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:00.862 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:49:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:01.035 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:01.119 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:01.183 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T23:49:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:49:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T23:49:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T23:49:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:49:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T23:49:02.249 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:49:02.250 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T23:49:02.250 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:49:02.250 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T23:49:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:02.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:49:02.319 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T23:49:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T23:49:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T23:49:02.946 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T23:49:02.996 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T23:49:03.036 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T23:49:03.079 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T23:49:03.121 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T23:49:03.317 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T23:49:03.360 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T23:49:03.390 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T23:49:03.420 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T23:49:03.463 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T23:49:03.474 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T23:49:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:49:03.519 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T23:49:03.520 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:49:03.520 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T23:49:03.520 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:49:03.520 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T23:49:03.520 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:03.560 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:49:03.585 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T23:49:03.585 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:49:03.585 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T23:49:03.585 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:03.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T23:49:03.661 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:03.978 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:49:04.003 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T23:49:04.004 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T23:49:04.004 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T23:49:04.004 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:49:04.004 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T23:49:04.004 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:04.058 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:04.097 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:04.117 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:04.943 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T23:49:05.036 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T23:49:05.085 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T23:49:05.170 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T23:49:05.205 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T23:49:05.231 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T23:49:05.265 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T23:49:05.265 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:05.349 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:05.349 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:05.433 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:05.458 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T23:49:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:05.635 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:05.636 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:05.719 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:05.792 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T23:49:06.861 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T23:49:06.862 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T23:49:06.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T23:49:06.875 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T23:49:06.876 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T23:49:06.877 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T23:49:06.878 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T23:49:06.879 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T23:49:06.880 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T23:49:06.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T23:49:06.882 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T23:49:06.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T23:49:06.884 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T23:49:06.885 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T23:49:06.885 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T23:49:06.885 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T23:49:06.885 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T23:49:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T23:49:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T23:49:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T23:49:08.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T23:49:09.014 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T23:49:09.219 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T23:49:09.260 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T23:49:09.291 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T23:49:09.320 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T23:49:09.366 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T23:49:09.735 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:09.735 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T23:49:09.736 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.737 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T23:49:09.738 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T23:49:09.739 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T23:49:09.740 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T23:49:09.741 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T23:49:09.742 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T23:49:09.743 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T23:49:09.744 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T23:49:09.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T23:49:09.746 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:49:09.747 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-15T23:49:10.013 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:10.055 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:10.078 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T23:49:10.163 DEBUG:teuthology.parallel:result is None 2024-09-15T23:49:10.994 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T23:49:11.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T23:49:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T23:49:11.221 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T23:49:11.262 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T23:49:11.296 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T23:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T23:49:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T23:49:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:11.498 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T23:49:11.523 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T23:49:11.530 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:11.530 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T23:49:11.692 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T23:49:11.693 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T23:49:11.694 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T23:49:11.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T23:49:11.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T23:49:11.707 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T23:49:11.708 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T23:49:11.709 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T23:49:11.710 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T23:49:11.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T23:49:11.712 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T23:49:11.713 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T23:49:11.714 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T23:49:11.715 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T23:49:11.716 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T23:49:11.716 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T23:49:11.716 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T23:49:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T23:49:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:11.810 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T23:49:11.878 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T23:49:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T23:49:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T23:49:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T23:49:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T23:49:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T23:49:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T23:49:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T23:49:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T23:49:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T23:49:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:49:14.310 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-15T23:49:14.759 DEBUG:teuthology.parallel:result is None 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T23:49:17.423 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T23:49:17.424 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T23:49:17.425 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T23:49:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T23:49:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T23:49:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T23:49:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T23:49:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T23:49:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T23:49:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T23:49:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T23:49:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T23:49:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T23:49:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T23:49:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T23:49:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T23:49:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T23:49:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T23:49:20.232 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.233 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T23:49:20.234 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.235 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T23:49:20.236 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T23:49:20.237 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T23:49:20.238 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T23:49:20.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T23:49:20.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T23:49:20.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T23:49:20.242 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:49:20.243 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-15T23:49:20.655 DEBUG:teuthology.parallel:result is None 2024-09-15T23:49:20.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:20.810 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T23:49:20.833 INFO:teuthology.orchestra.run.smithi031.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:20.834 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:20.834 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T23:49:20.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:20.975 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T23:49:21.003 INFO:teuthology.orchestra.run.smithi081.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:21.003 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:21.003 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T23:49:21.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:21.148 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T23:49:21.173 INFO:teuthology.orchestra.run.smithi102.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:21.173 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T23:49:21.174 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T23:49:21.175 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T23:49:21.175 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:49:21.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T23:49:21.206 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:49:21.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T23:49:21.239 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:49:21.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T23:49:21.270 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T23:49:21.270 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:49:21.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T23:49:21.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T23:49:21.368 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:49:21.368 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T23:49:21.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T23:49:21.468 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:49:21.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T23:49:21.498 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T23:49:21.567 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T23:49:21.568 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:49:21.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T23:49:21.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T23:49:21.667 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:49:21.667 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T23:49:21.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T23:49:21.764 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:49:21.764 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T23:49:21.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T23:49:21.861 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T23:49:21.862 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:49:21.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T23:49:21.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T23:49:21.962 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:49:21.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T23:49:21.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T23:49:22.061 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:49:22.061 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T23:49:22.091 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T23:49:22.159 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-15T23:49:22.254 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2}} 2024-09-15T23:49:22.254 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:22.255 INFO:tasks.cephadm:Cluster fsid is 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:49:22.255 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-15T23:49:22.255 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.81', 'mon.c': '172.21.15.102'} 2024-09-15T23:49:22.255 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-09-15T23:49:22.255 INFO:tasks.cephadm:First mgr is x 2024-09-15T23:49:22.255 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-15T23:49:22.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-09-15T23:49:22.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-09-15T23:49:22.316 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-09-15T23:49:22.342 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-15T23:49:22.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:22.489 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-15T23:49:22.643 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:49:22.644 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T23:49:22.644 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T23:49:22.645 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.013 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 23:49 /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.013 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.396 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 23:49 /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.397 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.767 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 23:49 /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.767 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.788 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.807 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T23:49:23.893 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-15T23:49:23.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T23:49:23.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T23:49:23.898 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T23:49:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T23:49:25.533 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T23:49:25.544 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-15T23:49:52.613 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-15T23:50:01.068 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-15T23:50:03.298 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-15T23:50:03.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-09-15T23:50:03.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-09-15T23:50:03.392 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-09-15T23:50:03.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-09-15T23:50:03.450 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-09-15T23:50:03.479 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-09-15T23:50:03.503 INFO:tasks.cephadm:Writing seed config... 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-15T23:50:03.505 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-15T23:50:03.509 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-15T23:50:03.510 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-15T23:50:03.511 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-15T23:50:03.512 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-15T23:50:03.512 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-15T23:50:03.512 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:50:03.512 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-15T23:50:03.531 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 21a99a0a-73bd-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-15T23:50:03.532 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service 2024-09-15T23:50:03.574 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x.service 2024-09-15T23:50:03.617 INFO:tasks.cephadm:Bootstrapping... 2024-09-15T23:50:03.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:50:03.832 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:50:03.832 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '21a99a0a-73bd-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-09-15T23:50:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-15T23:50:03.833 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-09-15T23:50:03.858 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-15T23:50:03.858 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-09-15T23:50:03.858 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-09-15T23:50:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T23:50:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T23:50:03.872 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T23:50:03.872 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:50:03.879 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-15T23:50:03.886 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-15T23:50:03.886 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-15T23:50:03.886 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-09-15T23:50:03.910 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-15T23:50:03.910 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-15T23:50:03.910 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-09-15T23:50:03.910 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-09-15T23:50:03.917 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T23:50:03.917 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T23:50:03.924 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T23:50:03.924 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:50:03.930 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-15T23:50:03.937 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140047427261440 on /run/cephadm/21a99a0a-73bd-11ef-bceb-c7b262605968.lock 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140047427261440 acquired on /run/cephadm/21a99a0a-73bd-11ef-bceb-c7b262605968.lock 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-09-15T23:50:03.938 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-15T23:50:03.941 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-09-15T23:50:03.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-09-15T23:50:03.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-15T23:50:03.944 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-15T23:50:03.945 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-09-15T23:50:03.947 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-15T23:50:03.947 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-15T23:50:03.947 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T23:50:03.947 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-15T23:50:03.947 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-15T23:50:03.948 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T23:50:03.949 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-15T23:50:03.949 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-15T23:50:03.949 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-15T23:50:03.949 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-15T23:50:03.949 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T23:50:07.540 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T23:50:07.541 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-15T23:50:16.276 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T23:50:16.276 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T23:50:16.276 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-09-15T23:50:17.190 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-09-15T23:50:17.190 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-09-15T23:50:18.227 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC5cudmG84tIRAACxJot97wBZ4HI7+W/IOpng== 2024-09-15T23:50:19.150 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC6cudmDer9IBAABttmDaXT1wa9lhnF+DwCOw== 2024-09-15T23:50:20.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQC7cudmZvDUGxAAZrAh43U95rLgizxQeLwPlw== 2024-09-15T23:50:20.141 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:50:21.071 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-09-15T23:50:22.510 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-09-15T23:50:22.730 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-15T23:50:22.872 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-15T23:50:23.048 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target. 2024-09-15T23:50:23.049 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target. 2024-09-15T23:50:23.283 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a 2024-09-15T23:50:23.283 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-15T23:50:23.463 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target.wants/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@.service. 2024-09-15T23:50:23.984 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T23:50:23.984 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-15T23:50:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T23:50:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:50:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-15T23:50:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T23:50:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-09-15T23:50:23.993 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-09-15T23:50:25.062 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.547098s) 2024-09-15T23:50:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.410+0000 7fec48c09640 1 Processor -- start 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.410+0000 7fec48c09640 1 -- start start 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.410+0000 7fec48c09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.410+0000 7fec48c09640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec4407be00 con 0x7fec44107f90 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.411+0000 7fec4259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.411+0000 7fec4259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39840/0 (socket says 172.21.15.31:39840) 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.411+0000 7fec4259c640 1 -- 172.21.15.31:0/3134807495 learned_addr learned my addr 172.21.15.31:0/3134807495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.411+0000 7fec4259c640 1 -- 172.21.15.31:0/3134807495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4410a8c0 con 0x7fec44107f90 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.413+0000 7fec4259c640 1 --2- 172.21.15.31:0/3134807495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec34009820 tx=0x7fec3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f19117b3154f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.413+0000 7fec41d9b640 1 -- 172.21.15.31:0/3134807495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec44107f90 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.413+0000 7fec41d9b640 1 -- 172.21.15.31:0/3134807495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34032440 con 0x7fec44107f90 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.413+0000 7fec41d9b640 1 -- 172.21.15.31:0/3134807495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec34032740 con 0x7fec44107f90 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 -- 172.21.15.31:0/3134807495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 msgr2=0x7fec4410a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 --2- 172.21.15.31:0/3134807495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec34009820 tx=0x7fec3402a270 comp rx=0 tx=0).stop 2024-09-15T23:50:25.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 -- 172.21.15.31:0/3134807495 shutdown_connections 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 --2- 172.21.15.31:0/3134807495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4410a380 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 -- 172.21.15.31:0/3134807495 >> 172.21.15.31:0/3134807495 conn(0x7fec44101bd0 msgr2=0x7fec44103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 -- 172.21.15.31:0/3134807495 shutdown_connections 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.414+0000 7fec48c09640 1 -- 172.21.15.31:0/3134807495 wait complete. 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec48c09640 1 Processor -- start 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec48c09640 1 -- start start 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec48c09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec48c09640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410ada0 con 0x7fec44107f90 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec4259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec4259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39852/0 (socket says 172.21.15.31:39852) 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec4259c640 1 -- 172.21.15.31:0/2028942937 learned_addr learned my addr 172.21.15.31:0/2028942937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:25.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec4259c640 1 -- 172.21.15.31:0/2028942937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4419b1b0 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.415+0000 7fec4259c640 1 --2- 172.21.15.31:0/2028942937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec340090b0 tx=0x7fec340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec34041070 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3403c7a0 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4419b440 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec34037040 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4419e130 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec34044070 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.416+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec3403d3f0 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.417+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec441083f0 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.419+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fec34046450 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.481+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fec4402d980 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.481+0000 7fec3b7fe640 1 -- 172.21.15.31:0/2028942937 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fec34046630 con 0x7fec44107f90 2024-09-15T23:50:25.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.482+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 msgr2=0x7fec4419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.482+0000 7fec48c09640 1 --2- 172.21.15.31:0/2028942937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec340090b0 tx=0x7fec340046c0 comp rx=0 tx=0).stop 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.482+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 shutdown_connections 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.482+0000 7fec48c09640 1 --2- 172.21.15.31:0/2028942937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec44107f90 0x7fec4419ac70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.482+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 >> 172.21.15.31:0/2028942937 conn(0x7fec44101bd0 msgr2=0x7fec44102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.483+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 shutdown_connections 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:24.483+0000 7fec48c09640 1 -- 172.21.15.31:0/2028942937 wait complete. 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-09-15T23:50:25.068 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-09-15T23:50:26.116 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:25 smithi031 ceph-mon[30833]: from='client.? 172.21.15.31:0/702622113' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T23:50:26.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T23:50:26.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.485+0000 7f36271ee640 1 Processor -- start 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.485+0000 7f36271ee640 1 -- start start 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.485+0000 7f36271ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.485+0000 7f36271ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3620106dd0 con 0x7f3620106400 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.486+0000 7f3624f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.486+0000 7f3624f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39854/0 (socket says 172.21.15.31:39854) 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.486+0000 7f3624f8a640 1 -- 172.21.15.31:0/436107737 learned_addr learned my addr 172.21.15.31:0/436107737 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.486+0000 7f3624f8a640 1 -- 172.21.15.31:0/436107737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36201075b0 con 0x7f3620106400 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.487+0000 7f3624f8a640 1 --2- 172.21.15.31:0/436107737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f361c009820 tx=0x7f361c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc87ec968d9494 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.487+0000 7f36177fe640 1 -- 172.21.15.31:0/436107737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c037070 con 0x7f3620106400 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.487+0000 7f36177fe640 1 -- 172.21.15.31:0/436107737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c032440 con 0x7f3620106400 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.487+0000 7f36177fe640 1 -- 172.21.15.31:0/436107737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c032740 con 0x7f3620106400 2024-09-15T23:50:26.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- 172.21.15.31:0/436107737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 msgr2=0x7f3620106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 --2- 172.21.15.31:0/436107737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f361c009820 tx=0x7f361c02a270 comp rx=0 tx=0).stop 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- 172.21.15.31:0/436107737 shutdown_connections 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 --2- 172.21.15.31:0/436107737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f3620106800 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- 172.21.15.31:0/436107737 >> 172.21.15.31:0/436107737 conn(0x7f3620101bb0 msgr2=0x7f3620103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- 172.21.15.31:0/436107737 shutdown_connections 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- 172.21.15.31:0/436107737 wait complete. 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 Processor -- start 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.488+0000 7f36271ee640 1 -- start start 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f36271ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f36271ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3620107b50 con 0x7f3620106400 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3624f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:26.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3624f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39862/0 (socket says 172.21.15.31:39862) 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3624f8a640 1 -- 172.21.15.31:0/702622113 learned_addr learned my addr 172.21.15.31:0/702622113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3624f8a640 1 -- 172.21.15.31:0/702622113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362018f4b0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3624f8a640 1 --2- 172.21.15.31:0/702622113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f361c0090b0 tx=0x7f361c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c041070 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c03c7a0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362018d6a0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.489+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c037040 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.490+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362018dbe0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.490+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f361c044070 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.490+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f361c03d3f0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.491+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36201081a0 con 0x7f3620106400 2024-09-15T23:50:26.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.493+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f361c046450 con 0x7f3620106400 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.553+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3620194570 con 0x7f3620106400 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.556+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f361c046630 con 0x7f3620106400 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.557+0000 7f3615ffb640 1 -- 172.21.15.31:0/702622113 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f361c056070 con 0x7f3620106400 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 msgr2=0x7f362018ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 --2- 172.21.15.31:0/702622113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f361c0090b0 tx=0x7f361c0046c0 comp rx=0 tx=0).stop 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 shutdown_connections 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 --2- 172.21.15.31:0/702622113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3620106400 0x7f362018ef70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 >> 172.21.15.31:0/702622113 conn(0x7f3620101bb0 msgr2=0x7f3620191930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 shutdown_connections 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:25.558+0000 7f36271ee640 1 -- 172.21.15.31:0/702622113 wait complete. 2024-09-15T23:50:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-09-15T23:50:27.163 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:26 smithi031 ceph-mon[30833]: from='client.? 172.21.15.31:0/2131821789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:50:27.165 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76ed5d6640 1 Processor -- start 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76ed5d6640 1 -- start start 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76ed5d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76ed5d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76e807be20 con 0x7f76e8107f70 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39864/0 (socket says 172.21.15.31:39864) 2024-09-15T23:50:27.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.523+0000 7f76e6ffd640 1 -- 172.21.15.31:0/1757728054 learned_addr learned my addr 172.21.15.31:0/1757728054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.524+0000 7f76e6ffd640 1 -- 172.21.15.31:0/1757728054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e810a8a0 con 0x7f76e8107f70 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.524+0000 7f76e6ffd640 1 --2- 172.21.15.31:0/1757728054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009820 tx=0x7f76d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7455315c38e459d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.524+0000 7f76e5ffb640 1 -- 172.21.15.31:0/1757728054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d8037070 con 0x7f76e8107f70 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.524+0000 7f76e5ffb640 1 -- 172.21.15.31:0/1757728054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76d80328d0 con 0x7f76e8107f70 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.524+0000 7f76e5ffb640 1 -- 172.21.15.31:0/1757728054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d80307d0 con 0x7f76e8107f70 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 -- 172.21.15.31:0/1757728054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 msgr2=0x7f76e810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 --2- 172.21.15.31:0/1757728054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009820 tx=0x7f76d802a270 comp rx=0 tx=0).stop 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 -- 172.21.15.31:0/1757728054 shutdown_connections 2024-09-15T23:50:27.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 --2- 172.21.15.31:0/1757728054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e810a360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 -- 172.21.15.31:0/1757728054 >> 172.21.15.31:0/1757728054 conn(0x7f76e8101bd0 msgr2=0x7f76e8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 -- 172.21.15.31:0/1757728054 shutdown_connections 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.525+0000 7f76ed5d6640 1 -- 172.21.15.31:0/1757728054 wait complete. 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76ed5d6640 1 Processor -- start 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76ed5d6640 1 -- start start 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76ed5d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76ed5d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810b880 con 0x7f76e8107f70 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39876/0 (socket says 172.21.15.31:39876) 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76e6ffd640 1 -- 172.21.15.31:0/2131821789 learned_addr learned my addr 172.21.15.31:0/2131821789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:27.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.526+0000 7f76e6ffd640 1 -- 172.21.15.31:0/2131821789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e819b970 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76e6ffd640 1 --2- 172.21.15.31:0/2131821789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76d8032e10 tx=0x7f76d8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d8042070 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76d8030c80 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e819bc00 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d8037040 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e819e8f0 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f76d8045070 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.527+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f76d8047a40 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.528+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d40052c0 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.530+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f76d803c6b0 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.590+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f76d4005880 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.590+0000 7f76cffff640 1 -- 172.21.15.31:0/2131821789 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f76d803c890 con 0x7f76e8107f70 2024-09-15T23:50:27.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 msgr2=0x7f76e819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 --2- 172.21.15.31:0/2131821789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76d8032e10 tx=0x7f76d8032e40 comp rx=0 tx=0).stop 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 shutdown_connections 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 --2- 172.21.15.31:0/2131821789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76e8107f70 0x7f76e819b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 >> 172.21.15.31:0/2131821789 conn(0x7f76e8101bd0 msgr2=0x7f76e81027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.591+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 shutdown_connections 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:26.592+0000 7f76ed5d6640 1 -- 172.21.15.31:0/2131821789 wait complete. 2024-09-15T23:50:27.174 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-09-15T23:50:27.452 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 systemd[1]: Stopping Ceph mon.a for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:50:27.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a[30829]: 2024-09-15T23:50:27.450+0000 7fbb1532f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:50:27.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a[30829]: 2024-09-15T23:50:27.450+0000 7fbb1532f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-15T23:50:27.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 podman[31078]: 2024-09-15 23:50:27.625223789 +0000 UTC m=+0.288789038 container died 106f97f3f550481fe400a0036e08192b43b24c480b0f0abed39c9e431841d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-15T23:50:28.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 podman[31078]: 2024-09-15 23:50:27.958646995 +0000 UTC m=+0.622212231 container cleanup 106f97f3f550481fe400a0036e08192b43b24c480b0f0abed39c9e431841d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T23:50:28.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:27 smithi031 bash[31078]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a 2024-09-15T23:50:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 podman[31090]: 2024-09-15 23:50:28.159651511 +0000 UTC m=+0.528348645 container remove 106f97f3f550481fe400a0036e08192b43b24c480b0f0abed39c9e431841d9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-15T23:50:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-15T23:50:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 systemd[1]: Stopped Ceph mon.a for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:50:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 systemd[1]: Starting Ceph mon.a for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:50:28.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 podman[31184]: 2024-09-15 23:50:28.508850218 +0000 UTC m=+0.016470277 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:50:28.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 podman[31184]: 2024-09-15 23:50:28.610551507 +0000 UTC m=+0.118171560 container create 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-15T23:50:28.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 podman[31184]: 2024-09-15 23:50:28.804511764 +0000 UTC m=+0.312131811 container init 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, ceph=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 podman[31184]: 2024-09-15 23:50:28.807716423 +0000 UTC m=+0.315336482 container start 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: pidfile_write: ignore empty --pid-file 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: load: jerasure load: lrc 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: RocksDB version: 7.9.2 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Git sha 0 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: DB SUMMARY 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: DB Session ID: E2DRUF4G9AXUL7811LU3 2024-09-15T23:50:28.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: CURRENT file: CURRENT 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104720 ; 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.error_if_exists: 0 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.create_if_missing: 0 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.paranoid_checks: 1 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.env: 0x560e8a5dd940 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T23:50:28.882 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.info_log: 0x560e8c1ab6e0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.statistics: (nil) 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.use_fsync: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_log_file_size: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_fallocate: 1 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.use_direct_reads: 0 2024-09-15T23:50:28.883 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.db_log_dir: 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.wal_dir: 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.write_buffer_manager: 0x560e8c1badc0 2024-09-15T23:50:28.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T23:50:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T23:50:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T23:50:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T23:50:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T23:50:28.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T23:50:28.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.unordered_write: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.row_cache: None 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.wal_filter: None 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.two_write_queues: 0 2024-09-15T23:50:28.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.wal_compression: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.atomic_flush: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.log_readahead_size: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T23:50:28.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_background_jobs: 2 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_background_compactions: -1 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_subcompactions: 1 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_open_files: -1 2024-09-15T23:50:28.889 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_background_flushes: -1 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Compression algorithms supported: 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kZSTD supported: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kXpressCompression supported: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kBZip2Compression supported: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kLZ4Compression supported: 1 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kZlibCompression supported: 1 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: kSnappyCompression supported: 1 2024-09-15T23:50:28.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.merge_operator: 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_filter: None 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e8c1ab320) 2024-09-15T23:50:28.891 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x560e8c19f1f0 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-09-15T23:50:28.892 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-09-15T23:50:28.893 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T23:50:28.894 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression: NoCompression 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.num_levels: 7 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T23:50:28.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T23:50:28.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T23:50:28.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T23:50:28.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T23:50:28.899 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T23:50:28.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T23:50:28.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T23:50:28.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T23:50:28.900 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T23:50:28.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.inplace_update_support: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.bloom_locality: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.max_successive_merges: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.ttl: 2592000 2024-09-15T23:50:28.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enable_blob_files: false 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.min_blob_size: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T23:50:28.903 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba20078a-51a4-4918-b098-c83877c013e3 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444228834450, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444228835459, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97718, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14155, "raw_average_key_size": 49, "raw_value_size": 90070, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726444228, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba20078a-51a4-4918-b098-c83877c013e3", "db_session_id": "E2DRUF4G9AXUL7811LU3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444228835558, "job": 1, "event": "recovery_finished"} 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560e8c1cce00 2024-09-15T23:50:28.904 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: DB pointer 0x560e8c2c8000 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T23:50:28.905 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 99.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 99.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T23:50:28.906 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 29.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 29.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x560e8c19f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???) e1 preinit fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).mds e1 new map 2024-09-15T23:50:28.907 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).mds e1 print_map 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: btime 2024-09-15T23:50:23:938450+0000 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-15T23:50:28.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 bash[31184]: 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 2024-09-15T23:50:28.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 systemd[1]: Started Ceph mon.a for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:50:28.909 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-15T23:50:29.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T23:50:29.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: monmap epoch 1 2024-09-15T23:50:29.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: last_changed 2024-09-15T23:50:20.496594+0000 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: min_mon_release 19 (squid) 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: election_strategy: 1 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: fsmap 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T23:50:29.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:28 smithi031 ceph-mon[31203]: mgrmap e1: no daemons active 2024-09-15T23:50:29.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.300+0000 7ff14675e640 1 Processor -- start 2024-09-15T23:50:29.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.300+0000 7ff14675e640 1 -- start start 2024-09-15T23:50:29.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.300+0000 7ff14675e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:29.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.300+0000 7ff14675e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff14007bde0 con 0x7ff140105da0 2024-09-15T23:50:29.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.301+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.301+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39892/0 (socket says 172.21.15.31:39892) 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.301+0000 7ff13ffff640 1 -- 172.21.15.31:0/2590133268 learned_addr learned my addr 172.21.15.31:0/2590133268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.301+0000 7ff13ffff640 1 -- 172.21.15.31:0/2590133268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401066e0 con 0x7ff140105da0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff13ffff640 1 --2- 172.21.15.31:0/2590133268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff130009820 tx=0x7ff13002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5762999eb749bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff13effd640 1 -- 172.21.15.31:0/2590133268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff130037070 con 0x7ff140105da0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff13effd640 1 -- 172.21.15.31:0/2590133268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff1300328d0 con 0x7ff140105da0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff13effd640 1 -- 172.21.15.31:0/2590133268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1300307d0 con 0x7ff140105da0 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff14675e640 1 -- 172.21.15.31:0/2590133268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 msgr2=0x7ff1401061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:29.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff14675e640 1 --2- 172.21.15.31:0/2590133268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff130009820 tx=0x7ff13002a270 comp rx=0 tx=0).stop 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.303+0000 7ff14675e640 1 -- 172.21.15.31:0/2590133268 shutdown_connections 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 --2- 172.21.15.31:0/2590133268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff1401061a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 -- 172.21.15.31:0/2590133268 >> 172.21.15.31:0/2590133268 conn(0x7ff140101bd0 msgr2=0x7ff140103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 -- 172.21.15.31:0/2590133268 shutdown_connections 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 -- 172.21.15.31:0/2590133268 wait complete. 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 Processor -- start 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.304+0000 7ff14675e640 1 -- start start 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff14675e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:29.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff14675e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff140107780 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39906/0 (socket says 172.21.15.31:39906) 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13ffff640 1 -- 172.21.15.31:0/265025285 learned_addr learned my addr 172.21.15.31:0/265025285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13ffff640 1 -- 172.21.15.31:0/265025285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019db60 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13ffff640 1 --2- 172.21.15.31:0/265025285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1300097f0 tx=0x7ff130030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff130037040 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff130032bb0 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019ddf0 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13003a390 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.305+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a0ae0 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.306+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff13003a580 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.306+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff130044ba0 con 0x7ff140105da0 2024-09-15T23:50:29.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.306+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1401061a0 con 0x7ff140105da0 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.309+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff130044400 con 0x7ff140105da0 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.368+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff14002d9b0 con 0x7ff140105da0 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.372+0000 7ff13d7fa640 1 -- 172.21.15.31:0/265025285 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff130032d60 con 0x7ff140105da0 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 msgr2=0x7ff14019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 --2- 172.21.15.31:0/265025285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff1300097f0 tx=0x7ff130030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 shutdown_connections 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 --2- 172.21.15.31:0/265025285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff140105da0 0x7ff14019d620 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 >> 172.21.15.31:0/265025285 conn(0x7ff140101bd0 msgr2=0x7ff1401942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 shutdown_connections 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:29.373+0000 7ff14675e640 1 -- 172.21.15.31:0/265025285 wait complete. 2024-09-15T23:50:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-15T23:50:30.004 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:50:30.004 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-09-15T23:50:30.004 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-15T23:50:30.004 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-15T23:50:30.260 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x 2024-09-15T23:50:30.260 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-15T23:50:30.414 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968.target.wants/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@.service. 2024-09-15T23:50:30.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:30 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/265025285' entity='client.admin' 2024-09-15T23:50:30.639 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 systemd[1]: Starting Ceph mgr.x for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:50:30.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 podman[31414]: 2024-09-15 23:50:30.638873957 +0000 UTC m=+0.104573648 container create b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-15T23:50:30.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 podman[31414]: 2024-09-15 23:50:30.550299088 +0000 UTC m=+0.015998747 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:50:30.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 podman[31414]: 2024-09-15 23:50:30.849280027 +0000 UTC m=+0.314979679 container init b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-15T23:50:30.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 podman[31414]: 2024-09-15 23:50:30.851982586 +0000 UTC m=+0.317682243 container start b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:50:30.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 bash[31414]: b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c 2024-09-15T23:50:30.939 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T23:50:30.939 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-15T23:50:30.947 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T23:50:30.947 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:50:30.947 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-15T23:50:30.948 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T23:50:30.956 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T23:50:30.956 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-15T23:50:30.964 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T23:50:30.964 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:50:30.965 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-15T23:50:30.965 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-15T23:50:30.965 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-09-15T23:50:30.965 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-09-15T23:50:31.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 systemd[1]: Started Ceph mgr.x for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:50:31.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:30 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:30.978+0000 7f107e55b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:50:31.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.039+0000 7f107e55b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:50:31.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.198+0000 7f107e55b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:50:31.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.268+0000 7f107e55b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:50:31.462 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.427+0000 7f107e55b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:50:31.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/577661059' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T23:50:31.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.589+0000 7f107e55b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:50:32.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.925+0000 7f107e55b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:50:32.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:31.989+0000 7f107e55b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:50:32.185 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:32 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:32.183+0000 7f107e55b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:50:32.501 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T23:50:32.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T23:50:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T23:50:23:938450+0000", 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T23:50:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T23:50:23.939128+0000", 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.390+0000 7fc8b41d7640 1 Processor -- start 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b41d7640 1 -- start start 2024-09-15T23:50:32.515 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b41d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b41d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac07bd50 con 0x7fc8ac108030 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b1f73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b1f73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39932/0 (socket says 172.21.15.31:39932) 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.391+0000 7fc8b1f73640 1 -- 172.21.15.31:0/1875708156 learned_addr learned my addr 172.21.15.31:0/1875708156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.392+0000 7fc8b1f73640 1 -- 172.21.15.31:0/1875708156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac10a960 con 0x7fc8ac108030 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.392+0000 7fc8b1f73640 1 --2- 172.21.15.31:0/1875708156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009820 tx=0x7fc8a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b8a328eae094ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.392+0000 7fc8b0f71640 1 -- 172.21.15.31:0/1875708156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a8037070 con 0x7fc8ac108030 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b0f71640 1 -- 172.21.15.31:0/1875708156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc8a80328d0 con 0x7fc8ac108030 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b0f71640 1 -- 172.21.15.31:0/1875708156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a80307d0 con 0x7fc8ac108030 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 -- 172.21.15.31:0/1875708156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 msgr2=0x7fc8ac10a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 --2- 172.21.15.31:0/1875708156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009820 tx=0x7fc8a802a270 comp rx=0 tx=0).stop 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 -- 172.21.15.31:0/1875708156 shutdown_connections 2024-09-15T23:50:32.516 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 --2- 172.21.15.31:0/1875708156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac10a420 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 -- 172.21.15.31:0/1875708156 >> 172.21.15.31:0/1875708156 conn(0x7fc8ac101bb0 msgr2=0x7fc8ac103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 -- 172.21.15.31:0/1875708156 shutdown_connections 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.393+0000 7fc8b41d7640 1 -- 172.21.15.31:0/1875708156 wait complete. 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b41d7640 1 Processor -- start 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b41d7640 1 -- start start 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b41d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b41d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac10b880 con 0x7fc8ac108030 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b1f73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b1f73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39936/0 (socket says 172.21.15.31:39936) 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.394+0000 7fc8b1f73640 1 -- 172.21.15.31:0/577661059 learned_addr learned my addr 172.21.15.31:0/577661059 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc8b1f73640 1 -- 172.21.15.31:0/577661059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac19db80 con 0x7fc8ac108030 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc8b1f73640 1 --2- 172.21.15.31:0/577661059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8032e10 tx=0x7fc8a8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a8042070 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc8a8030c80 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac19de10 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a8037040 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a0b00 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8a8045070 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.395+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8a8047a80 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.396+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac108490 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.398+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc8a803c700 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.460+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc8ac1a0cd0 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.460+0000 7fc89effd640 1 -- 172.21.15.31:0/577661059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc8a803c8e0 con 0x7fc8ac108030 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 msgr2=0x7fc8ac19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 --2- 172.21.15.31:0/577661059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8032e10 tx=0x7fc8a8032e40 comp rx=0 tx=0).stop 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 shutdown_connections 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 --2- 172.21.15.31:0/577661059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8ac108030 0x7fc8ac19d640 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:32.518 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 >> 172.21.15.31:0/577661059 conn(0x7fc8ac101bb0 msgr2=0x7fc8ac102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:32.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 shutdown_connections 2024-09-15T23:50:32.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:31.462+0000 7fc8b41d7640 1 -- 172.21.15.31:0/577661059 wait complete. 2024-09-15T23:50:32.519 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-09-15T23:50:33.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:33 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:33.115+0000 7f107e55b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:50:33.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:33 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:33.185+0000 7f107e55b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:50:34.218 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:33 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:33.946+0000 7f107e55b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:50:34.218 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.067+0000 7f107e55b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:50:34.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.216+0000 7f107e55b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:50:34.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.279+0000 7f107e55b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:50:34.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.436+0000 7f107e55b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:50:34.811 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.507+0000 7f107e55b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:50:34.811 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.573+0000 7f107e55b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:50:34.811 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.741+0000 7f107e55b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:50:34.811 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.809+0000 7f107e55b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:50:35.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:35 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2170903253' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T23:50:35.100 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.881+0000 7f107e55b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:50:35.100 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:34.970+0000 7f107e55b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:50:35.100 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:35.035+0000 7f107e55b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:50:35.100 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:35.098+0000 7f107e55b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:50:35.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:35.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:50:35.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:50:35.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T23:50:35.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-15T23:50:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T23:50:35.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T23:50:35.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T23:50:35.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T23:50:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T23:50:35.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T23:50:35.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T23:50:35.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T23:50:23:938450+0000", 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:35.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T23:50:23.939128+0000", 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.904+0000 7f1eb5cc0640 1 Processor -- start 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eb5cc0640 1 -- start start 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eb5cc0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:35.889 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eb5cc0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb007bd90 con 0x7f1eb0105de0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37522/0 (socket says 172.21.15.31:37522) 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eaf7fe640 1 -- 172.21.15.31:0/2132128213 learned_addr learned my addr 172.21.15.31:0/2132128213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.905+0000 7f1eaf7fe640 1 -- 172.21.15.31:0/2132128213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0106720 con 0x7f1eb0105de0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.906+0000 7f1eaf7fe640 1 --2- 172.21.15.31:0/2132128213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009820 tx=0x7f1ea002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a469ec4bbc68e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.906+0000 7f1eaeffd640 1 -- 172.21.15.31:0/2132128213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ea0037070 con 0x7f1eb0105de0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.906+0000 7f1eaeffd640 1 -- 172.21.15.31:0/2132128213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1ea00328d0 con 0x7f1eb0105de0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.906+0000 7f1eaeffd640 1 -- 172.21.15.31:0/2132128213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ea00307d0 con 0x7f1eb0105de0 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2132128213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 msgr2=0x7f1eb01061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 --2- 172.21.15.31:0/2132128213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009820 tx=0x7f1ea002a270 comp rx=0 tx=0).stop 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2132128213 shutdown_connections 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 --2- 172.21.15.31:0/2132128213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb01061e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:35.890 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2132128213 >> 172.21.15.31:0/2132128213 conn(0x7f1eb0101bb0 msgr2=0x7f1eb0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2132128213 shutdown_connections 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.907+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2132128213 wait complete. 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eb5cc0640 1 Processor -- start 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eb5cc0640 1 -- start start 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eb5cc0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eb5cc0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0107640 con 0x7f1eb0105de0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37532/0 (socket says 172.21.15.31:37532) 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eaf7fe640 1 -- 172.21.15.31:0/2170903253 learned_addr learned my addr 172.21.15.31:0/2170903253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eaf7fe640 1 -- 172.21.15.31:0/2170903253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb019db20 con 0x7f1eb0105de0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.908+0000 7f1eaf7fe640 1 --2- 172.21.15.31:0/2170903253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00097f0 tx=0x7f1ea0030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ea0037040 con 0x7f1eb0105de0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1ea0032bb0 con 0x7f1eb0105de0 2024-09-15T23:50:35.891 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb019ddb0 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ea003a390 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a0aa0 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ea003a600 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.909+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ea0043360 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.910+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb0107d10 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.912+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1ea0032d60 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.974+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1eb01970d0 con 0x7f1eb0105de0 2024-09-15T23:50:35.892 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.975+0000 7f1ead7fa640 1 -- 172.21.15.31:0/2170903253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1ea003aa80 con 0x7f1eb0105de0 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 msgr2=0x7f1eb019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 --2- 172.21.15.31:0/2170903253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00097f0 tx=0x7f1ea0030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 shutdown_connections 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 --2- 172.21.15.31:0/2170903253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb0105de0 0x7f1eb019d5e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 >> 172.21.15.31:0/2170903253 conn(0x7f1eb0101bb0 msgr2=0x7f1eb01942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 shutdown_connections 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:34.976+0000 7f1eb5cc0640 1 -- 172.21.15.31:0/2170903253 wait complete. 2024-09-15T23:50:35.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-09-15T23:50:36.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:35.934+0000 7f107e55b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: Activating manager daemon x 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: mgrmap e2: x(active, starting, since 0.00294657s) 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: Manager daemon x is now available 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T23:50:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T23:50:36.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' 2024-09-15T23:50:36.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' 2024-09-15T23:50:36.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:36 smithi031 ceph-mon[31203]: from='mgr.14100 172.21.15.31:0/258432732' entity='mgr.x' 2024-09-15T23:50:38.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:37 smithi031 ceph-mon[31203]: mgrmap e3: x(active, since 1.0076s) 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T23:50:39.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T23:50:39.172 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T23:50:23:938450+0000", 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T23:50:39.173 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T23:50:23.939128+0000", 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T23:50:39.174 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.294+0000 7fc4f33b2640 1 Processor -- start 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f33b2640 1 -- start start 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f33b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f33b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec07bd90 con 0x7fc4ec105de0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f114e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f114e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37594/0 (socket says 172.21.15.31:37594) 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f114e640 1 -- 172.21.15.31:0/2257489512 learned_addr learned my addr 172.21.15.31:0/2257489512 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.295+0000 7fc4f114e640 1 -- 172.21.15.31:0/2257489512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec106720 con 0x7fc4ec105de0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.296+0000 7fc4f114e640 1 --2- 172.21.15.31:0/2257489512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009820 tx=0x7fc4e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23a2cef49d35655 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.296+0000 7fc4e3fff640 1 -- 172.21.15.31:0/2257489512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e8037070 con 0x7fc4ec105de0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.296+0000 7fc4e3fff640 1 -- 172.21.15.31:0/2257489512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc4e80328d0 con 0x7fc4ec105de0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.296+0000 7fc4e3fff640 1 -- 172.21.15.31:0/2257489512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e80307d0 con 0x7fc4ec105de0 2024-09-15T23:50:39.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.297+0000 7fc4f33b2640 1 -- 172.21.15.31:0/2257489512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 msgr2=0x7fc4ec1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.297+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/2257489512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009820 tx=0x7fc4e802a270 comp rx=0 tx=0).stop 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.297+0000 7fc4f33b2640 1 -- 172.21.15.31:0/2257489512 shutdown_connections 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.297+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/2257489512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec1061e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.297+0000 7fc4f33b2640 1 -- 172.21.15.31:0/2257489512 >> 172.21.15.31:0/2257489512 conn(0x7fc4ec101bb0 msgr2=0x7fc4ec103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.298+0000 7fc4f33b2640 1 -- 172.21.15.31:0/2257489512 shutdown_connections 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.298+0000 7fc4f33b2640 1 -- 172.21.15.31:0/2257489512 wait complete. 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.298+0000 7fc4f33b2640 1 Processor -- start 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.298+0000 7fc4f33b2640 1 -- start start 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f33b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f33b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec107640 con 0x7fc4ec105de0 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f114e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f114e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37610/0 (socket says 172.21.15.31:37610) 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f114e640 1 -- 172.21.15.31:0/3768854817 learned_addr learned my addr 172.21.15.31:0/3768854817 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:39.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f114e640 1 -- 172.21.15.31:0/3768854817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19db20 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f114e640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8032e10 tx=0x7fc4e8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e8042070 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc4e8030c80 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19ddb0 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e8037040 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.299+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1a0aa0 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.300+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc4e8045070 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.300+0000 7fc4e27fc640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 0x7fc4d803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.301+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4e806c140 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.301+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec107d10 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.301+0000 7fc4f094d640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 0x7fc4d803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.301+0000 7fc4f094d640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 0x7fc4d803b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0098c0 tx=0x7fc4dc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.306+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4e8045360 con 0x7fc4ec105de0 2024-09-15T23:50:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.552+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc4ec02da30 con 0x7fc4ec105de0 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.552+0000 7fc4e27fc640 1 -- 172.21.15.31:0/3768854817 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc4e802ed90 con 0x7fc4ec105de0 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 msgr2=0x7fc4d803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 0x7fc4d803b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0098c0 tx=0x7fc4dc006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 msgr2=0x7fc4ec19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8032e10 tx=0x7fc4e8032e40 comp rx=0 tx=0).stop 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 shutdown_connections 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7fc4d8039160 0x7fc4d803b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 --2- 172.21.15.31:0/3768854817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec105de0 0x7fc4ec19d5e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.554+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 >> 172.21.15.31:0/3768854817 conn(0x7fc4ec101bb0 msgr2=0x7fc4ec1942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.555+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 shutdown_connections 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:38.555+0000 7fc4f33b2640 1 -- 172.21.15.31:0/3768854817 wait complete. 2024-09-15T23:50:39.178 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-09-15T23:50:39.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:38 smithi031 ceph-mon[31203]: mgrmap e4: x(active, since 2s) 2024-09-15T23:50:39.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:38 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3768854817' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T23:50:40.194 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:39 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3112436329' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T23:50:40.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.635+0000 7f2ccaede640 1 Processor -- start 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.635+0000 7f2ccaede640 1 -- start start 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.635+0000 7f2ccaede640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.635+0000 7f2ccaede640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4106dd0 con 0x7f2cc4106400 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.636+0000 7f2cc8c7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:40.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.636+0000 7f2cc8c7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37620/0 (socket says 172.21.15.31:37620) 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.636+0000 7f2cc8c7a640 1 -- 172.21.15.31:0/3348299388 learned_addr learned my addr 172.21.15.31:0/3348299388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.636+0000 7f2cc8c7a640 1 -- 172.21.15.31:0/3348299388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc41075b0 con 0x7f2cc4106400 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2cc8c7a640 1 --2- 172.21.15.31:0/3348299388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009820 tx=0x7f2cb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaf824a306ae8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2cc37fe640 1 -- 172.21.15.31:0/3348299388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb4037070 con 0x7f2cc4106400 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2cc37fe640 1 -- 172.21.15.31:0/3348299388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2cb40328d0 con 0x7f2cc4106400 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2cc37fe640 1 -- 172.21.15.31:0/3348299388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb40307d0 con 0x7f2cc4106400 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2ccaede640 1 -- 172.21.15.31:0/3348299388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 msgr2=0x7f2cc4106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.637+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3348299388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009820 tx=0x7f2cb402a270 comp rx=0 tx=0).stop 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- 172.21.15.31:0/3348299388 shutdown_connections 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3348299388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc4106800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- 172.21.15.31:0/3348299388 >> 172.21.15.31:0/3348299388 conn(0x7f2cc4101bb0 msgr2=0x7f2cc4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- 172.21.15.31:0/3348299388 shutdown_connections 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- 172.21.15.31:0/3348299388 wait complete. 2024-09-15T23:50:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 Processor -- start 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- start start 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2ccaede640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc410a960 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2cc8c7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2cc8c7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37626/0 (socket says 172.21.15.31:37626) 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.638+0000 7f2cc8c7a640 1 -- 172.21.15.31:0/3112436329 learned_addr learned my addr 172.21.15.31:0/3112436329 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2cc8c7a640 1 -- 172.21.15.31:0/3112436329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc419b980 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2cc8c7a640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb4037040 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2cb4032bb0 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc419bc10 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb403a390 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.639+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc419e900 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.640+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2cb403a580 con 0x7f2cc4106400 2024-09-15T23:50:40.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.640+0000 7f2cc1ffb640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 0x7f2cac03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.640+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc410aae0 con 0x7f2cc4106400 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.640+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2cb406c420 con 0x7f2cc4106400 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.640+0000 7f2cc3fff640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 0x7f2cac03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.641+0000 7f2cc3fff640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 0x7f2cac03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80098c0 tx=0x7f2cb8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.645+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cb4040210 con 0x7f2cc4106400 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.820+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2cc4106880 con 0x7f2cc4106400 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.821+0000 7f2cc1ffb640 1 -- 172.21.15.31:0/3112436329 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f2cb4031560 con 0x7f2cc4106400 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 msgr2=0x7f2cac03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 0x7f2cac03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80098c0 tx=0x7f2cb8006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 msgr2=0x7f2cc419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb4030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 shutdown_connections 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f2cac0391b0 0x7f2cac03b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 --2- 172.21.15.31:0/3112436329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc4106400 0x7f2cc419b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 >> 172.21.15.31:0/3112436329 conn(0x7f2cc4101bb0 msgr2=0x7f2cc4102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:40.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 shutdown_connections 2024-09-15T23:50:40.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:39.824+0000 7f2ccaede640 1 -- 172.21.15.31:0/3112436329 wait complete. 2024-09-15T23:50:40.484 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-09-15T23:50:41.401 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/315091034' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-15T23:50:41.417 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setuser ceph since I am not root 2024-09-15T23:50:41.417 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setgroup ceph since I am not root 2024-09-15T23:50:41.417 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.340+0000 7f2761c4c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:50:41.417 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.399+0000 7f2761c4c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:50:41.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.554+0000 7f2761c4c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:50:41.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.623+0000 7f2761c4c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:50:41.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.909+0000 7ff2f8302640 1 Processor -- start 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.909+0000 7ff2f8302640 1 -- start start 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f8302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f8302640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f007c860 con 0x7ff2f007de30 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f609e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f609e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37638/0 (socket says 172.21.15.31:37638) 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f609e640 1 -- 172.21.15.31:0/2856488999 learned_addr learned my addr 172.21.15.31:0/2856488999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.910+0000 7ff2f609e640 1 -- 172.21.15.31:0/2856488999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f007c9e0 con 0x7ff2f007de30 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.911+0000 7ff2f609e640 1 --2- 172.21.15.31:0/2856488999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009820 tx=0x7ff2ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1446946bfa8868c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:41.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.911+0000 7ff2f509c640 1 -- 172.21.15.31:0/2856488999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec037070 con 0x7ff2f007de30 2024-09-15T23:50:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.911+0000 7ff2f509c640 1 -- 172.21.15.31:0/2856488999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff2ec0328d0 con 0x7ff2f007de30 2024-09-15T23:50:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.911+0000 7ff2f509c640 1 -- 172.21.15.31:0/2856488999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec0307d0 con 0x7ff2f007de30 2024-09-15T23:50:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 -- 172.21.15.31:0/2856488999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 msgr2=0x7ff2f007c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 --2- 172.21.15.31:0/2856488999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009820 tx=0x7ff2ec02a270 comp rx=0 tx=0).stop 2024-09-15T23:50:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 -- 172.21.15.31:0/2856488999 shutdown_connections 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 --2- 172.21.15.31:0/2856488999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f007c290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 -- 172.21.15.31:0/2856488999 >> 172.21.15.31:0/2856488999 conn(0x7ff2f0102c70 msgr2=0x7ff2f0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 -- 172.21.15.31:0/2856488999 shutdown_connections 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 -- 172.21.15.31:0/2856488999 wait complete. 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.912+0000 7ff2f8302640 1 Processor -- start 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f8302640 1 -- start start 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f8302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f8302640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f010da90 con 0x7ff2f007de30 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f609e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f609e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37652/0 (socket says 172.21.15.31:37652) 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f609e640 1 -- 172.21.15.31:0/315091034 learned_addr learned my addr 172.21.15.31:0/315091034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f609e640 1 -- 172.21.15.31:0/315091034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f019db60 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f609e640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0097f0 tx=0x7ff2ec030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec037040 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff2ec032bb0 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.913+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f019ddf0 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.914+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec03a390 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.914+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01a0ae0 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.914+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7ff2ec03a580 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.914+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f007c310 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.915+0000 7ff2e77fe640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 0x7ff2dc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.915+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2ec06c680 con 0x7ff2f007de30 2024-09-15T23:50:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.915+0000 7ff2f589d640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 0x7ff2dc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.915+0000 7ff2f589d640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 0x7ff2dc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098c0 tx=0x7ff2e0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:40.920+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2ec032dd0 con 0x7ff2f007de30 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.140+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff2f01a1160 con 0x7ff2f007de30 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.191+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff2ec031560 con 0x7ff2f007de30 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.193+0000 7ff2e77fe640 1 -- 172.21.15.31:0/315091034 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7ff2ec049cf0 con 0x7ff2f007de30 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 msgr2=0x7ff2dc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 0x7ff2dc03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098c0 tx=0x7ff2e0006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 msgr2=0x7ff2f019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0097f0 tx=0x7ff2ec030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 shutdown_connections 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7ff2dc0391b0 0x7ff2dc03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 --2- 172.21.15.31:0/315091034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2f007de30 0x7ff2f019d620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:41.780 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 >> 172.21.15.31:0/315091034 conn(0x7ff2f0102c70 msgr2=0x7ff2f0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:41.781 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 shutdown_connections 2024-09-15T23:50:41.781 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:41.195+0000 7ff2f8302640 1 -- 172.21.15.31:0/315091034 wait complete. 2024-09-15T23:50:42.096 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.778+0000 7f2761c4c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:50:42.097 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:41.927+0000 7f2761c4c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:50:42.382 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:42 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/315091034' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-15T23:50:42.382 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:42 smithi031 ceph-mon[31203]: mgrmap e5: x(active, since 5s) 2024-09-15T23:50:42.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:42.262+0000 7f2761c4c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:50:42.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:42.326+0000 7f2761c4c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:50:42.700 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:42.523+0000 7f2761c4c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:50:42.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:50:42.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-15T23:50:42.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T23:50:42.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecf664640 1 Processor -- start 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecf664640 1 -- start start 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecf664640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecf664640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8106dd0 con 0x7f8ec8106400 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39734/0 (socket says 172.21.15.31:39734) 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.174+0000 7f8ecd400640 1 -- 172.21.15.31:0/1462384591 learned_addr learned my addr 172.21.15.31:0/1462384591 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.175+0000 7f8ecd400640 1 -- 172.21.15.31:0/1462384591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec81075b0 con 0x7f8ec8106400 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.175+0000 7f8ecd400640 1 --2- 172.21.15.31:0/1462384591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009820 tx=0x7f8ec402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff640d2905abe29a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ebbfff640 1 -- 172.21.15.31:0/1462384591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4037070 con 0x7f8ec8106400 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ebbfff640 1 -- 172.21.15.31:0/1462384591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ec40328d0 con 0x7f8ec8106400 2024-09-15T23:50:42.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ebbfff640 1 -- 172.21.15.31:0/1462384591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec40307d0 con 0x7f8ec8106400 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ecf664640 1 -- 172.21.15.31:0/1462384591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 msgr2=0x7f8ec8106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1462384591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009820 tx=0x7f8ec402a270 comp rx=0 tx=0).stop 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ecf664640 1 -- 172.21.15.31:0/1462384591 shutdown_connections 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1462384591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec8106800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.176+0000 7f8ecf664640 1 -- 172.21.15.31:0/1462384591 >> 172.21.15.31:0/1462384591 conn(0x7f8ec8101bb0 msgr2=0x7f8ec8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 -- 172.21.15.31:0/1462384591 shutdown_connections 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 -- 172.21.15.31:0/1462384591 wait complete. 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 Processor -- start 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 -- start start 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.177+0000 7f8ecf664640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec810a960 con 0x7f8ec8106400 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39738/0 (socket says 172.21.15.31:39738) 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecd400640 1 -- 172.21.15.31:0/1597616917 learned_addr learned my addr 172.21.15.31:0/1597616917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:42.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecd400640 1 -- 172.21.15.31:0/1597616917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec819b980 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecd400640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009940 tx=0x7f8ec4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4042070 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ec4030c80 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4037040 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec819bc10 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.178+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec819e900 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f8ec4045070 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec810aae0 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8eba7fc640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 0x7f8eb403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ec406c210 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8eccbff640 1 -- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 msgr2=0x7f8eb403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.179+0000 7f8eccbff640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 0x7f8eb403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.184+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ec4045360 con 0x7f8ec8106400 2024-09-15T23:50:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.380+0000 7f8eccbff640 1 -- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 msgr2=0x7f8eb403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.380+0000 7f8eccbff640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 0x7f8eb403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.408+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8ec80008d0 con 0x7f8ec8106400 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.408+0000 7f8eba7fc640 1 -- 172.21.15.31:0/1597616917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f8ec403ae00 con 0x7f8ec8106400 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 msgr2=0x7f8eb403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 0x7f8eb403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 msgr2=0x7f8ec819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009940 tx=0x7f8ec4032e10 comp rx=0 tx=0).stop 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 shutdown_connections 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f8eb4039200 0x7f8eb403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 --2- 172.21.15.31:0/1597616917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ec8106400 0x7f8ec819b440 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 >> 172.21.15.31:0/1597616917 conn(0x7f8ec8101bb0 msgr2=0x7f8ec8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 shutdown_connections 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:42.411+0000 7f8ecf664640 1 -- 172.21.15.31:0/1597616917 wait complete. 2024-09-15T23:50:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-15T23:50:42.990 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-09-15T23:50:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:43 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1597616917' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T23:50:43.592 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:43 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:43.465+0000 7f2761c4c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:50:43.592 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:43 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:43.536+0000 7f2761c4c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:50:44.555 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.284+0000 7f2761c4c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:50:44.556 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.404+0000 7f2761c4c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:50:44.846 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.553+0000 7f2761c4c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:50:44.847 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.617+0000 7f2761c4c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:50:44.847 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.773+0000 7f2761c4c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:50:44.847 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.844+0000 7f2761c4c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:50:45.143 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:44.907+0000 7f2761c4c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:50:45.143 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.073+0000 7f2761c4c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:50:45.143 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.141+0000 7f2761c4c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:50:45.423 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.211+0000 7f2761c4c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:50:45.423 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.297+0000 7f2761c4c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:50:45.423 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.359+0000 7f2761c4c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:50:45.423 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:45.421+0000 7f2761c4c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:50:46.504 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:46.248+0000 7f2761c4c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:50:46.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: Active manager daemon x restarted 2024-09-15T23:50:46.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: Activating manager daemon x 2024-09-15T23:50:46.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: osdmap e2: 0 total, 0 up, 0 in 2024-09-15T23:50:46.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: mgrmap e6: x(active, starting, since 0.00349874s) 2024-09-15T23:50:46.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: Manager daemon x is now available 2024-09-15T23:50:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:50:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:50:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T23:50:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:46 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19ee283640 1 Processor -- start 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19ee283640 1 -- start start 2024-09-15T23:50:47.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19ee283640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19ee283640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19e807be10 con 0x7f19e8107f70 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39744/0 (socket says 172.21.15.31:39744) 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.383+0000 7f19e7fff640 1 -- 172.21.15.31:0/1304220509 learned_addr learned my addr 172.21.15.31:0/1304220509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.384+0000 7f19e7fff640 1 -- 172.21.15.31:0/1304220509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e810a8a0 con 0x7f19e8107f70 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.384+0000 7f19e7fff640 1 --2- 172.21.15.31:0/1304220509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009820 tx=0x7f19d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5259053d5b6b5758 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.384+0000 7f19e6ffd640 1 -- 172.21.15.31:0/1304220509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d8037070 con 0x7f19e8107f70 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19e6ffd640 1 -- 172.21.15.31:0/1304220509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f19d80328d0 con 0x7f19e8107f70 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19e6ffd640 1 -- 172.21.15.31:0/1304220509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d80307d0 con 0x7f19e8107f70 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 -- 172.21.15.31:0/1304220509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 msgr2=0x7f19e810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 --2- 172.21.15.31:0/1304220509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009820 tx=0x7f19d802a270 comp rx=0 tx=0).stop 2024-09-15T23:50:47.843 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 -- 172.21.15.31:0/1304220509 shutdown_connections 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 --2- 172.21.15.31:0/1304220509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e810a360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 -- 172.21.15.31:0/1304220509 >> 172.21.15.31:0/1304220509 conn(0x7f19e8101bb0 msgr2=0x7f19e8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 -- 172.21.15.31:0/1304220509 shutdown_connections 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.385+0000 7f19ee283640 1 -- 172.21.15.31:0/1304220509 wait complete. 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19ee283640 1 Processor -- start 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19ee283640 1 -- start start 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19ee283640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19ee283640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19e810b880 con 0x7f19e8107f70 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39748/0 (socket says 172.21.15.31:39748) 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.386+0000 7f19e7fff640 1 -- 172.21.15.31:0/2396306862 learned_addr learned my addr 172.21.15.31:0/2396306862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19e7fff640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e819b9d0 con 0x7f19e8107f70 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19e7fff640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19d8032e10 tx=0x7f19d8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:47.844 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d8042070 con 0x7f19e8107f70 2024-09-15T23:50:47.845 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f19d8030c80 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e819bc60 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d8037040 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.387+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e819e950 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.388+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f19d8045070 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.389+0000 7f19e57fa640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.389+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19d003bdd0 con 0x7f19d0039200 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.389+0000 7f19e77fe640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 msgr2=0x7f19d003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.389+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19d806c140 con 0x7f19e8107f70 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.389+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.590+0000 7f19e77fe640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 msgr2=0x7f19d003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.590+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:50:47.848 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.991+0000 7f19e77fe640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 msgr2=0x7f19d003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:43.991+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:44.792+0000 7f19e77fe640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 msgr2=0x7f19d003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1530295855 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:44.792+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:46.252+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f19d806ad50 con 0x7f19e8107f70 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:46.252+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 msgr2=0x7f19d003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:46.252+0000 7f19e57fa640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.253+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f19d803c5f0 con 0x7f19e8107f70 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.253+0000 7f19e57fa640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 0x7f19d003ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.253+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19d003bdd0 con 0x7f19d003cb30 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.254+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 0x7f19d003ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.255+0000 7f19e77fe640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 0x7f19d003ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19dc003980 tx=0x7f19dc0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.255+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f19d003bdd0 con 0x7f19d003cb30 2024-09-15T23:50:47.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f19e81083d0 con 0x7f19d003cb30 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19e57fa640 1 -- 172.21.15.31:0/2396306862 <== mgr.14118 v2:172.21.15.31:6800/639707976 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f19e81083d0 con 0x7f19d003cb30 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 msgr2=0x7f19d003ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19ee283640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 0x7f19d003ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19dc003980 tx=0x7f19dc0092b0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 msgr2=0x7f19e819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.268+0000 7f19ee283640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19d8032e10 tx=0x7f19d8032e40 comp rx=0 tx=0).stop 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 shutdown_connections 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f19d003cb30 0x7f19d003ef20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:6800/1530295855,v1:172.21.15.31:6801/1530295855] conn(0x7f19d0039200 0x7f19d003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 --2- 172.21.15.31:0/2396306862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e8107f70 0x7f19e819b490 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 >> 172.21.15.31:0/2396306862 conn(0x7f19e8101bb0 msgr2=0x7f19e8102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 shutdown_connections 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:47.269+0000 7f19ee283640 1 -- 172.21.15.31:0/2396306862 wait complete. 2024-09-15T23:50:47.850 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-09-15T23:50:47.851 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-09-15T23:50:48.225 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:48 smithi031 ceph-mon[31203]: Found migration_current of "None". Setting to last migration. 2024-09-15T23:50:48.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:48 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:48.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:48 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:48.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:48 smithi031 ceph-mon[31203]: mgrmap e7: x(active, since 1.00569s) 2024-09-15T23:50:49.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.224+0000 7f0f1988d640 1 Processor -- start 2024-09-15T23:50:49.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.224+0000 7f0f1988d640 1 -- start start 2024-09-15T23:50:49.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f1988d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:49.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f1988d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14107e90 con 0x7f0f141074c0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39824/0 (socket says 172.21.15.31:39824) 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f12ffd640 1 -- 172.21.15.31:0/2968839948 learned_addr learned my addr 172.21.15.31:0/2968839948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.225+0000 7f0f12ffd640 1 -- 172.21.15.31:0/2968839948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141086c0 con 0x7f0f141074c0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.226+0000 7f0f12ffd640 1 --2- 172.21.15.31:0/2968839948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009820 tx=0x7f0f0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38343bc0fc4884 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.226+0000 7f0f127fc640 1 -- 172.21.15.31:0/2968839948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04037070 con 0x7f0f141074c0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.226+0000 7f0f127fc640 1 -- 172.21.15.31:0/2968839948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0f040328d0 con 0x7f0f141074c0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.226+0000 7f0f127fc640 1 -- 172.21.15.31:0/2968839948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f040307d0 con 0x7f0f141074c0 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 -- 172.21.15.31:0/2968839948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 msgr2=0x7f0f141078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2968839948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009820 tx=0x7f0f0402a270 comp rx=0 tx=0).stop 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 -- 172.21.15.31:0/2968839948 shutdown_connections 2024-09-15T23:50:49.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2968839948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f141078c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 -- 172.21.15.31:0/2968839948 >> 172.21.15.31:0/2968839948 conn(0x7f0f14102c70 msgr2=0x7f0f14105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 -- 172.21.15.31:0/2968839948 shutdown_connections 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.227+0000 7f0f1988d640 1 -- 172.21.15.31:0/2968839948 wait complete. 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f1988d640 1 Processor -- start 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f1988d640 1 -- start start 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f1988d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f1988d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410ba70 con 0x7f0f141074c0 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39832/0 (socket says 172.21.15.31:39832) 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f12ffd640 1 -- 172.21.15.31:0/2549513482 learned_addr learned my addr 172.21.15.31:0/2549513482 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.228+0000 7f0f12ffd640 1 -- 172.21.15.31:0/2549513482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1419b9f0 con 0x7f0f141074c0 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f12ffd640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f040097f0 tx=0x7f0f04030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04037040 con 0x7f0f141074c0 2024-09-15T23:50:49.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0f04032bb0 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1419bc80 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1419e970 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.229+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f0403a390 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.230+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f0f0403a5d0 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.230+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40052c0 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.230+0000 7f0f1888b640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 0x7f0f0003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.230+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f0406c440 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.230+0000 7f0f0bfff640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 0x7f0f0003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.231+0000 7f0f0bfff640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 0x7f0f0003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0098c0 tx=0x7f0efc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.235+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f040313a0 con 0x7f0f141074c0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.427+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0ef4000bf0 con 0x7f0f000390a0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.432+0000 7f0f1888b640 1 -- 172.21.15.31:0/2549513482 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ef4000bf0 con 0x7f0f000390a0 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 msgr2=0x7f0f0003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:49.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 0x7f0f0003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0098c0 tx=0x7f0efc006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 msgr2=0x7f0f1419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f040097f0 tx=0x7f0f04030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 shutdown_connections 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f0f000390a0 0x7f0f0003b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 --2- 172.21.15.31:0/2549513482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f141074c0 0x7f0f1419b4b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 >> 172.21.15.31:0/2549513482 conn(0x7f0f14102c70 msgr2=0x7f0f141037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 shutdown_connections 2024-09-15T23:50:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:48.435+0000 7f0f1988d640 1 -- 172.21.15.31:0/2549513482 wait complete. 2024-09-15T23:50:49.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-mon[31203]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T23:50:49.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-mon[31203]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T23:50:49.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:49.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:49.347+0000 7f26c8f66640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:23:50:49] ENGINE Error in HTTPServer.serve 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Traceback (most recent call last): 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._connections.run(self.expiration_interval) 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._run(expiration_interval) 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T23:50:49.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s = self.context.wrap_socket( 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: return self.sslsocket_class._create( 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.do_handshake() 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._sslobj.do_handshake() 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T23:50:49.631 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:50:50.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:50 smithi031 ceph-mon[31203]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:50.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:50 smithi031 ceph-mon[31203]: [15/Sep/2024:23:50:49] ENGINE Bus STARTING 2024-09-15T23:50:50.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:50 smithi031 ceph-mon[31203]: mgrmap e8: x(active, since 2s) 2024-09-15T23:50:50.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:50 smithi031 ceph-mon[31203]: [15/Sep/2024:23:50:49] ENGINE Serving on http://172.21.15.31:8765 2024-09-15T23:50:50.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:50 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:50:50.272 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-15T23:50:50.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d906b640 1 Processor -- start 2024-09-15T23:50:50.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d906b640 1 -- start start 2024-09-15T23:50:50.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d906b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:50.273 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d906b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4106dd0 con 0x7f09d4106400 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39838/0 (socket says 172.21.15.31:39838) 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.440+0000 7f09d2d9d640 1 -- 172.21.15.31:0/1481072171 learned_addr learned my addr 172.21.15.31:0/1481072171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.441+0000 7f09d2d9d640 1 -- 172.21.15.31:0/1481072171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41075b0 con 0x7f09d4106400 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.441+0000 7f09d2d9d640 1 --2- 172.21.15.31:0/1481072171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009820 tx=0x7f09c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bb7ed7681b5405 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d259c640 1 -- 172.21.15.31:0/1481072171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c4037070 con 0x7f09d4106400 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d259c640 1 -- 172.21.15.31:0/1481072171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f09c40328d0 con 0x7f09d4106400 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d259c640 1 -- 172.21.15.31:0/1481072171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c40307d0 con 0x7f09d4106400 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 -- 172.21.15.31:0/1481072171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 msgr2=0x7f09d4106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 --2- 172.21.15.31:0/1481072171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009820 tx=0x7f09c402a270 comp rx=0 tx=0).stop 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 -- 172.21.15.31:0/1481072171 shutdown_connections 2024-09-15T23:50:50.275 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 --2- 172.21.15.31:0/1481072171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d4106800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:50.276 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 -- 172.21.15.31:0/1481072171 >> 172.21.15.31:0/1481072171 conn(0x7f09d4101bb0 msgr2=0x7f09d4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:50.276 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.442+0000 7f09d906b640 1 -- 172.21.15.31:0/1481072171 shutdown_connections 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.443+0000 7f09d906b640 1 -- 172.21.15.31:0/1481072171 wait complete. 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.443+0000 7f09d906b640 1 Processor -- start 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.443+0000 7f09d906b640 1 -- start start 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.443+0000 7f09d906b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.443+0000 7f09d906b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f09d407af80 con 0x7f09d4106400 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39842/0 (socket says 172.21.15.31:39842) 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d2d9d640 1 -- 172.21.15.31:0/899681086 learned_addr learned my addr 172.21.15.31:0/899681086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d2d9d640 1 -- 172.21.15.31:0/899681086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d410c7b0 con 0x7f09d4106400 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d2d9d640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009940 tx=0x7f09c4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c4042070 con 0x7f09d4106400 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d410a9a0 con 0x7f09d4106400 2024-09-15T23:50:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f09c4030c80 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09c4037040 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.444+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d410aee0 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.445+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f09c4045070 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.445+0000 7f09d0d99640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 0x7f09c003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.445+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d4106880 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.446+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09c406c200 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.446+0000 7f09cbfff640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 0x7f09c003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.446+0000 7f09cbfff640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 0x7f09c003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0098c0 tx=0x7f09bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.450+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09c4045360 con 0x7f09d4106400 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.628+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f09d4194db0 con 0x7f09c00391b0 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.629+0000 7f09d0d99640 1 -- 172.21.15.31:0/899681086 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f09d4194db0 con 0x7f09c00391b0 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 msgr2=0x7f09c003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 0x7f09c003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0098c0 tx=0x7f09bc006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 msgr2=0x7f09d410c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009940 tx=0x7f09c4032e10 comp rx=0 tx=0).stop 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 shutdown_connections 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f09c00391b0 0x7f09c003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 --2- 172.21.15.31:0/899681086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09d4106400 0x7f09d410c270 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 >> 172.21.15.31:0/899681086 conn(0x7f09d4101bb0 msgr2=0x7f09d41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 shutdown_connections 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:49.631+0000 7f09d906b640 1 -- 172.21.15.31:0/899681086 wait complete. 2024-09-15T23:50:50.279 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-09-15T23:50:51.112 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Generating public/private rsa key pair. 2024-09-15T23:50:51.112 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Your identification has been saved in /tmp/tmp3hk66i1n/key 2024-09-15T23:50:51.112 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Your public key has been saved in /tmp/tmp3hk66i1n/key.pub 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: The key fingerprint is: 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: SHA256:f48lksGhV1kpHvdQ76erRMIcpUOO9ITnCyn/WZsC8cU ceph-21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: The key's randomart image is: 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: +---[RSA 3072]----+ 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | ..o . .o.| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | ..*.ooo+ .| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | .+Boo+ o.| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | . +=.=E ..| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | oS+Bo. o| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | ooo=. ..| 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | o+o+oo | 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | ++o= . | 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: | .o.o | 2024-09-15T23:50:51.113 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: +----[SHA256]-----+ 2024-09-15T23:50:51.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: [15/Sep/2024:23:50:49] ENGINE Serving on https://172.21.15.31:7150 2024-09-15T23:50:51.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: [15/Sep/2024:23:50:49] ENGINE Bus STARTED 2024-09-15T23:50:51.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: [15/Sep/2024:23:50:49] ENGINE Error in HTTPServer.serve 2024-09-15T23:50:51.409 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-15T23:50:51.409 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-15T23:50:51.410 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:51.411 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:51 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.664+0000 7f506b91f640 1 Processor -- start 2024-09-15T23:50:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.664+0000 7f506b91f640 1 -- start start 2024-09-15T23:50:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f506b91f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f506b91f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f506407c8a0 con 0x7f506407de70 2024-09-15T23:50:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f50696bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f50696bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39844/0 (socket says 172.21.15.31:39844) 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f50696bb640 1 -- 172.21.15.31:0/2370848490 learned_addr learned my addr 172.21.15.31:0/2370848490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.665+0000 7f50696bb640 1 -- 172.21.15.31:0/2370848490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506407ca20 con 0x7f506407de70 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.666+0000 7f50696bb640 1 --2- 172.21.15.31:0/2370848490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5060009820 tx=0x7f506002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46ec240956e9289 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.666+0000 7f5057fff640 1 -- 172.21.15.31:0/2370848490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5060037070 con 0x7f506407de70 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.666+0000 7f5057fff640 1 -- 172.21.15.31:0/2370848490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f50600328d0 con 0x7f506407de70 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.666+0000 7f5057fff640 1 -- 172.21.15.31:0/2370848490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50600307d0 con 0x7f506407de70 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- 172.21.15.31:0/2370848490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 msgr2=0x7f506407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 --2- 172.21.15.31:0/2370848490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5060009820 tx=0x7f506002a270 comp rx=0 tx=0).stop 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- 172.21.15.31:0/2370848490 shutdown_connections 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 --2- 172.21.15.31:0/2370848490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506407c2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- 172.21.15.31:0/2370848490 >> 172.21.15.31:0/2370848490 conn(0x7f5064102c70 msgr2=0x7f5064105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- 172.21.15.31:0/2370848490 shutdown_connections 2024-09-15T23:50:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- 172.21.15.31:0/2370848490 wait complete. 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 Processor -- start 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.667+0000 7f506b91f640 1 -- start start 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f506b91f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f506b91f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f506410dab0 con 0x7f506407de70 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50696bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50696bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39848/0 (socket says 172.21.15.31:39848) 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50696bb640 1 -- 172.21.15.31:0/1874368317 learned_addr learned my addr 172.21.15.31:0/1874368317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50696bb640 1 -- 172.21.15.31:0/1874368317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506419dbb0 con 0x7f506407de70 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50696bb640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5060009940 tx=0x7f5060032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5060042070 con 0x7f506407de70 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5060030c80 con 0x7f506407de70 2024-09-15T23:50:51.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506419de40 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5060037040 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.668+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a0b30 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.669+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5060045070 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.669+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506407c350 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.669+0000 7f50567fc640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 0x7f505003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.670+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f506006c150 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.670+0000 7f5068eba640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 0x7f505003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.670+0000 7f5068eba640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 0x7f505003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50580098c0 tx=0x7f5058006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.674+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5060045360 con 0x7f506407de70 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:50.854+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5064103ee0 con 0x7f5050039200 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.108+0000 7f50567fc640 1 -- 172.21.15.31:0/1874368317 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5064103ee0 con 0x7f5050039200 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 msgr2=0x7f505003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 0x7f505003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50580098c0 tx=0x7f5058006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 msgr2=0x7f506419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5060009940 tx=0x7f5060032e10 comp rx=0 tx=0).stop 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 shutdown_connections 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5050039200 0x7f505003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 --2- 172.21.15.31:0/1874368317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506407de70 0x7f506419d670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 >> 172.21.15.31:0/1874368317 conn(0x7f5064102c70 msgr2=0x7f50641037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 shutdown_connections 2024-09-15T23:50:51.696 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:51.111+0000 7f506b91f640 1 -- 172.21.15.31:0/1874368317 wait complete. 2024-09-15T23:50:52.299 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:52 smithi031 ceph-mon[31203]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:52.299 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:52 smithi031 ceph-mon[31203]: Generating ssh key... 2024-09-15T23:50:52.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:50:52.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885eb6a640 1 Processor -- start 2024-09-15T23:50:52.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885eb6a640 1 -- start start 2024-09-15T23:50:52.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885eb6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:52.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885eb6a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f885807bde0 con 0x7f8858107fb0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885c906640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885c906640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43142/0 (socket says 172.21.15.31:43142) 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885c906640 1 -- 172.21.15.31:0/3208922700 learned_addr learned my addr 172.21.15.31:0/3208922700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.107+0000 7f885c906640 1 -- 172.21.15.31:0/3208922700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885810a8e0 con 0x7f8858107fb0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.108+0000 7f885c906640 1 --2- 172.21.15.31:0/3208922700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8840009820 tx=0x7f884002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7a804b411233c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.108+0000 7f88577fe640 1 -- 172.21.15.31:0/3208922700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8840037070 con 0x7f8858107fb0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f88577fe640 1 -- 172.21.15.31:0/3208922700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f88400328d0 con 0x7f8858107fb0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f88577fe640 1 -- 172.21.15.31:0/3208922700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88400307d0 con 0x7f8858107fb0 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 -- 172.21.15.31:0/3208922700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 msgr2=0x7f885810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 --2- 172.21.15.31:0/3208922700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8840009820 tx=0x7f884002a270 comp rx=0 tx=0).stop 2024-09-15T23:50:52.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 -- 172.21.15.31:0/3208922700 shutdown_connections 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 --2- 172.21.15.31:0/3208922700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885810a3a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 -- 172.21.15.31:0/3208922700 >> 172.21.15.31:0/3208922700 conn(0x7f8858101bd0 msgr2=0x7f8858103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 -- 172.21.15.31:0/3208922700 shutdown_connections 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.109+0000 7f885eb6a640 1 -- 172.21.15.31:0/3208922700 wait complete. 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885eb6a640 1 Processor -- start 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885eb6a640 1 -- start start 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885eb6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885eb6a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f885810b860 con 0x7f8858107fb0 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885c906640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885c906640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43158/0 (socket says 172.21.15.31:43158) 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.110+0000 7f885c906640 1 -- 172.21.15.31:0/2311557267 learned_addr learned my addr 172.21.15.31:0/2311557267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:52.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f885c906640 1 -- 172.21.15.31:0/2311557267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885819b970 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f885c906640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88400097f0 tx=0x7f8840030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8840037040 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8840032bb0 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885819bc00 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f884003a390 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.111+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885819e8f0 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.112+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f884003b3f0 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.112+0000 7f8855ffb640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 0x7f884403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.112+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8858108410 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.112+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f884006c530 con 0x7f8858107fb0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.112+0000 7f8857fff640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 0x7f884403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:52.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.113+0000 7f8857fff640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 0x7f884403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f884800ad30 tx=0x7f88480093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.117+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8840032d60 con 0x7f8858107fb0 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.297+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8858102ea0 con 0x7f88440391b0 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.298+0000 7f8855ffb640 1 -- 172.21.15.31:0/2311557267 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8858102ea0 con 0x7f88440391b0 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 msgr2=0x7f884403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 0x7f884403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f884800ad30 tx=0x7f88480093f0 comp rx=0 tx=0).stop 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 msgr2=0x7f885819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88400097f0 tx=0x7f8840030d00 comp rx=0 tx=0).stop 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 shutdown_connections 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f88440391b0 0x7f884403b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 --2- 172.21.15.31:0/2311557267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8858107fb0 0x7f885819b430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 >> 172.21.15.31:0/2311557267 conn(0x7f8858101bd0 msgr2=0x7f8858102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:52.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 shutdown_connections 2024-09-15T23:50:52.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:52.300+0000 7f885eb6a640 1 -- 172.21.15.31:0/2311557267 wait complete. 2024-09-15T23:50:52.886 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-15T23:50:52.886 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-09-15T23:50:52.888 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-09-15T23:50:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:53 smithi031 ceph-mon[31203]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:54.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:54 smithi031 ceph-mon[31203]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:55.455 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:55 smithi031 ceph-mon[31203]: Deploying cephadm binary to smithi031 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.310+0000 7fa9cd802640 1 Processor -- start 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.310+0000 7fa9cd802640 1 -- start start 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.310+0000 7fa9cd802640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.310+0000 7fa9cd802640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8107e90 con 0x7fa9c81074c0 2024-09-15T23:50:58.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.311+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.311+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43160/0 (socket says 172.21.15.31:43160) 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.311+0000 7fa9c6ffd640 1 -- 172.21.15.31:0/1619411588 learned_addr learned my addr 172.21.15.31:0/1619411588 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.311+0000 7fa9c6ffd640 1 -- 172.21.15.31:0/1619411588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c81086c0 con 0x7fa9c81074c0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9c6ffd640 1 --2- 172.21.15.31:0/1619411588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009820 tx=0x7fa9b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2860a90c2272ae2f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9c5ffb640 1 -- 172.21.15.31:0/1619411588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b8037070 con 0x7fa9c81074c0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9c5ffb640 1 -- 172.21.15.31:0/1619411588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa9b80328d0 con 0x7fa9c81074c0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9c5ffb640 1 -- 172.21.15.31:0/1619411588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b80307d0 con 0x7fa9c81074c0 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 -- 172.21.15.31:0/1619411588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 msgr2=0x7fa9c81078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 --2- 172.21.15.31:0/1619411588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009820 tx=0x7fa9b802a270 comp rx=0 tx=0).stop 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 -- 172.21.15.31:0/1619411588 shutdown_connections 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 --2- 172.21.15.31:0/1619411588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c81078c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 -- 172.21.15.31:0/1619411588 >> 172.21.15.31:0/1619411588 conn(0x7fa9c8102c70 msgr2=0x7fa9c8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:58.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 -- 172.21.15.31:0/1619411588 shutdown_connections 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.312+0000 7fa9cd802640 1 -- 172.21.15.31:0/1619411588 wait complete. 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.313+0000 7fa9cd802640 1 Processor -- start 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.313+0000 7fa9cd802640 1 -- start start 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cd802640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cd802640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810ba70 con 0x7fa9c81074c0 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43174/0 (socket says 172.21.15.31:43174) 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9c6ffd640 1 -- 172.21.15.31:0/297718329 learned_addr learned my addr 172.21.15.31:0/297718329 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9c6ffd640 1 -- 172.21.15.31:0/297718329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c807e060 con 0x7fa9c81074c0 2024-09-15T23:50:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9c6ffd640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009940 tx=0x7fa9b8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b8042070 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa9b8030c80 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c807c250 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b8037040 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.314+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c807c790 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.315+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa9b8045070 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.316+0000 7fa9cc800640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 0x7fa9b003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.316+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c8107940 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.316+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9b806c500 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.316+0000 7fa9c67fc640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 0x7fa9b003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.316+0000 7fa9c67fc640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 0x7fa9b003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0098c0 tx=0x7fa9bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.320+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9b8045360 con 0x7fa9c81074c0 2024-09-15T23:50:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:53.503+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) -- 0x7fa9c8103ec0 con 0x7fa9b0039200 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.411+0000 7fa9cc800640 1 -- 172.21.15.31:0/297718329 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa9c8103ec0 con 0x7fa9b0039200 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 msgr2=0x7fa9b003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 0x7fa9b003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0098c0 tx=0x7fa9bc006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 msgr2=0x7fa9c807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009940 tx=0x7fa9b8032e10 comp rx=0 tx=0).stop 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 shutdown_connections 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fa9b0039200 0x7fa9b003b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 --2- 172.21.15.31:0/297718329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c81074c0 0x7fa9c807db20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 >> 172.21.15.31:0/297718329 conn(0x7fa9c8102c70 msgr2=0x7fa9c81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 shutdown_connections 2024-09-15T23:50:58.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:57.414+0000 7fa9cd802640 1 -- 172.21.15.31:0/297718329 wait complete. 2024-09-15T23:50:58.010 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-09-15T23:50:58.691 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:58 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:50:58.691 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:58 smithi031 ceph-mon[31203]: Added host smithi031 2024-09-15T23:50:58.691 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:58 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.460+0000 7f5116a86640 1 Processor -- start 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.460+0000 7f5116a86640 1 -- start start 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.460+0000 7f5116a86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.460+0000 7f5116a86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f511007bdb0 con 0x7f5110107fd0 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.461+0000 7f5114822640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.461+0000 7f5114822640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43178/0 (socket says 172.21.15.31:43178) 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.461+0000 7f5114822640 1 -- 172.21.15.31:0/1967525760 learned_addr learned my addr 172.21.15.31:0/1967525760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:59.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.461+0000 7f5114822640 1 -- 172.21.15.31:0/1967525760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511010a900 con 0x7f5110107fd0 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.462+0000 7f5114822640 1 --2- 172.21.15.31:0/1967525760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5108009e80 tx=0x7f510802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a25213fd261f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.462+0000 7f510f7fe640 1 -- 172.21.15.31:0/1967525760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5108036070 con 0x7f5110107fd0 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.462+0000 7f510f7fe640 1 -- 172.21.15.31:0/1967525760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5108031890 con 0x7f5110107fd0 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.462+0000 7f510f7fe640 1 -- 172.21.15.31:0/1967525760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5108007980 con 0x7f5110107fd0 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- 172.21.15.31:0/1967525760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 msgr2=0x7f511010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 --2- 172.21.15.31:0/1967525760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5108009e80 tx=0x7f510802a7a0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- 172.21.15.31:0/1967525760 shutdown_connections 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 --2- 172.21.15.31:0/1967525760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511010a3c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- 172.21.15.31:0/1967525760 >> 172.21.15.31:0/1967525760 conn(0x7f5110101bb0 msgr2=0x7f5110103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- 172.21.15.31:0/1967525760 shutdown_connections 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- 172.21.15.31:0/1967525760 wait complete. 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 Processor -- start 2024-09-15T23:50:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- start start 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.463+0000 7f5116a86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f511010b880 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5114822640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5114822640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43194/0 (socket says 172.21.15.31:43194) 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5114822640 1 -- 172.21.15.31:0/549271341 learned_addr learned my addr 172.21.15.31:0/549271341 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5114822640 1 -- 172.21.15.31:0/549271341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511019b930 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5114822640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5108031c70 tx=0x7f5108031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5108041020 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f510803c900 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511019bbc0 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5108036040 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.464+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511019e8b0 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.465+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5108044020 con 0x7f5110107fd0 2024-09-15T23:50:59.558 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.465+0000 7f510dffb640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 0x7f50f803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.466+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f510806d6d0 con 0x7f5110107fd0 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.466+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5110108430 con 0x7f5110107fd0 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.466+0000 7f510ffff640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 0x7f50f803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.466+0000 7f510ffff640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 0x7f50f803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51040098c0 tx=0x7f5104006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.470+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f510804b020 con 0x7f5110107fd0 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.690+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5110102e80 con 0x7f50f8039200 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.695+0000 7f510dffb640 1 -- 172.21.15.31:0/549271341 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5110102e80 con 0x7f50f8039200 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 msgr2=0x7f50f803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 0x7f50f803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51040098c0 tx=0x7f5104006db0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 msgr2=0x7f511019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5108031c70 tx=0x7f5108031ca0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 shutdown_connections 2024-09-15T23:50:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f50f8039200 0x7f50f803b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 --2- 172.21.15.31:0/549271341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5110107fd0 0x7f511019b3f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:50:59.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 >> 172.21.15.31:0/549271341 conn(0x7f5110101bb0 msgr2=0x7f5110102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:50:59.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 shutdown_connections 2024-09-15T23:50:59.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:50:58.697+0000 7f5116a86640 1 -- 172.21.15.31:0/549271341 wait complete. 2024-09-15T23:50:59.560 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-09-15T23:50:59.799 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:59 smithi031 ceph-mon[31203]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:50:59.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:59 smithi031 ceph-mon[31203]: Saving service mon spec with placement count:5 2024-09-15T23:50:59.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:50:59 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:01.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.882+0000 7f4bb557f640 1 Processor -- start 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.882+0000 7f4bb557f640 1 -- start start 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.882+0000 7f4bb557f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.882+0000 7f4bb557f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0107e70 con 0x7f4bb01074a0 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.883+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.883+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43208/0 (socket says 172.21.15.31:43208) 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.883+0000 7f4baeffd640 1 -- 172.21.15.31:0/3780373816 learned_addr learned my addr 172.21.15.31:0/3780373816 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.883+0000 7f4baeffd640 1 -- 172.21.15.31:0/3780373816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb01086a0 con 0x7f4bb01074a0 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4baeffd640 1 --2- 172.21.15.31:0/3780373816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009820 tx=0x7f4ba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c06525c95fb2d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4badffb640 1 -- 172.21.15.31:0/3780373816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba0037070 con 0x7f4bb01074a0 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4badffb640 1 -- 172.21.15.31:0/3780373816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba00328f0 con 0x7f4bb01074a0 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4badffb640 1 -- 172.21.15.31:0/3780373816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba0030740 con 0x7f4bb01074a0 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4bb557f640 1 -- 172.21.15.31:0/3780373816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 msgr2=0x7f4bb01078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.884+0000 7f4bb557f640 1 --2- 172.21.15.31:0/3780373816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009820 tx=0x7f4ba002a270 comp rx=0 tx=0).stop 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 -- 172.21.15.31:0/3780373816 shutdown_connections 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 --2- 172.21.15.31:0/3780373816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb01078a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 -- 172.21.15.31:0/3780373816 >> 172.21.15.31:0/3780373816 conn(0x7f4bb0102c90 msgr2=0x7f4bb01050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 -- 172.21.15.31:0/3780373816 shutdown_connections 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 -- 172.21.15.31:0/3780373816 wait complete. 2024-09-15T23:51:01.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 Processor -- start 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.885+0000 7f4bb557f640 1 -- start start 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4bb557f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4bb557f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010ba50 con 0x7f4bb01074a0 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43224/0 (socket says 172.21.15.31:43224) 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4baeffd640 1 -- 172.21.15.31:0/2125002279 learned_addr learned my addr 172.21.15.31:0/2125002279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4baeffd640 1 -- 172.21.15.31:0/2125002279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb019b9e0 con 0x7f4bb01074a0 2024-09-15T23:51:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4baeffd640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00097f0 tx=0x7f4ba0030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba0042070 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba0030d80 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb019bc70 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ba0037040 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.886+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb019e960 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.887+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4ba0045070 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.887+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb010bbd0 con 0x7f4bb01074a0 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.888+0000 7f4b97fff640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 0x7f4b9803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:01.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.888+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ba006c440 con 0x7f4bb01074a0 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.888+0000 7f4bae7fc640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 0x7f4b9803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.888+0000 7f4bae7fc640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 0x7f4b9803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098c0 tx=0x7f4ba4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:00.892+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ba003add0 con 0x7f4bb01074a0 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.140+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4bb0103f50 con 0x7f4b98039200 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.146+0000 7f4b97fff640 1 -- 172.21.15.31:0/2125002279 <== mgr.14118 v2:172.21.15.31:6800/639707976 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4bb0103f50 con 0x7f4b98039200 2024-09-15T23:51:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 msgr2=0x7f4b9803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 0x7f4b9803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40098c0 tx=0x7f4ba4006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 msgr2=0x7f4bb019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00097f0 tx=0x7f4ba0030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 shutdown_connections 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f4b98039200 0x7f4b9803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 --2- 172.21.15.31:0/2125002279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bb01074a0 0x7f4bb019b4a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:01.774 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 >> 172.21.15.31:0/2125002279 conn(0x7f4bb0102c90 msgr2=0x7f4bb0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:01.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 shutdown_connections 2024-09-15T23:51:01.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:01.149+0000 7f4bb557f640 1 -- 172.21.15.31:0/2125002279 wait complete. 2024-09-15T23:51:02.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:01 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:02.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:01 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:02.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:01 smithi031 ceph-mon[31203]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:02.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:01 smithi031 ceph-mon[31203]: Saving service mgr spec with placement count:2 2024-09-15T23:51:02.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:01 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:03 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1105890327' entity='client.admin' 2024-09-15T23:51:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:03 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.165+0000 7fd2d249c640 1 Processor -- start 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.165+0000 7fd2d249c640 1 -- start start 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2d249c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2d249c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10a630 con 0x7fd2cc07bca0 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57650/0 (socket says 172.21.15.31:57650) 2024-09-15T23:51:03.552 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2cbfff640 1 -- 172.21.15.31:0/3098490840 learned_addr learned my addr 172.21.15.31:0/3098490840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.166+0000 7fd2cbfff640 1 -- 172.21.15.31:0/3098490840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc10a7b0 con 0x7fd2cc07bca0 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.167+0000 7fd2cbfff640 1 --2- 172.21.15.31:0/3098490840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009820 tx=0x7fd2bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460025cc4f639f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.167+0000 7fd2cb7fe640 1 -- 172.21.15.31:0/3098490840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc037070 con 0x7fd2cc07bca0 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.167+0000 7fd2cb7fe640 1 -- 172.21.15.31:0/3098490840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2bc0328f0 con 0x7fd2cc07bca0 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.167+0000 7fd2cb7fe640 1 -- 172.21.15.31:0/3098490840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc030740 con 0x7fd2cc07bca0 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- 172.21.15.31:0/3098490840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 msgr2=0x7fd2cc10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 --2- 172.21.15.31:0/3098490840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009820 tx=0x7fd2bc02a270 comp rx=0 tx=0).stop 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- 172.21.15.31:0/3098490840 shutdown_connections 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 --2- 172.21.15.31:0/3098490840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc07bca0 0x7fd2cc10a060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- 172.21.15.31:0/3098490840 >> 172.21.15.31:0/3098490840 conn(0x7fd2cc101bb0 msgr2=0x7fd2cc103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- 172.21.15.31:0/3098490840 shutdown_connections 2024-09-15T23:51:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- 172.21.15.31:0/3098490840 wait complete. 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 Processor -- start 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- start start 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.168+0000 7fd2d249c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10b890 con 0x7fd2cc107890 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57654/0 (socket says 172.21.15.31:57654) 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2cbfff640 1 -- 172.21.15.31:0/1105890327 learned_addr learned my addr 172.21.15.31:0/1105890327 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2cbfff640 1 -- 172.21.15.31:0/1105890327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc107cb0 con 0x7fd2cc107890 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2cbfff640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097f0 tx=0x7fd2bc030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc042070 con 0x7fd2cc107890 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2bc030d80 con 0x7fd2cc107890 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc106540 con 0x7fd2cc107890 2024-09-15T23:51:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc037040 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.169+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc106a20 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.170+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd2bc045070 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.170+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc107e80 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.170+0000 7fd2c9ffb640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 0x7fd2b803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.171+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd2bc06c410 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.171+0000 7fd2c3fff640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 0x7fd2b803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.171+0000 7fd2c3fff640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 0x7fd2b803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098c0 tx=0x7fd2b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.175+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2bc06bd60 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.369+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd2cc07bd40 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.371+0000 7fd2c9ffb640 1 -- 172.21.15.31:0/1105890327 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd2bc03d430 con 0x7fd2cc107890 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 msgr2=0x7fd2b803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 0x7fd2b803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098c0 tx=0x7fd2b4006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:03.555 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 msgr2=0x7fd2cc106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097f0 tx=0x7fd2bc030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 shutdown_connections 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7fd2b80391b0 0x7fd2b803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 --2- 172.21.15.31:0/1105890327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2cc107890 0x7fd2cc106000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 >> 172.21.15.31:0/1105890327 conn(0x7fd2cc101bb0 msgr2=0x7fd2cc1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 shutdown_connections 2024-09-15T23:51:03.556 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:02.374+0000 7fd2d249c640 1 -- 172.21.15.31:0/1105890327 wait complete. 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.947+0000 7f525442d640 1 Processor -- start 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.947+0000 7f525442d640 1 -- start start 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f525442d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f525442d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f524c07be00 con 0x7f524c107f80 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f52521c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f52521c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57666/0 (socket says 172.21.15.31:57666) 2024-09-15T23:51:04.759 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f52521c9640 1 -- 172.21.15.31:0/1053032054 learned_addr learned my addr 172.21.15.31:0/1053032054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.948+0000 7f52521c9640 1 -- 172.21.15.31:0/1053032054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c10a8b0 con 0x7f524c107f80 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.949+0000 7f52521c9640 1 --2- 172.21.15.31:0/1053032054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5240009820 tx=0x7f524002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109d05e5a7687788 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.950+0000 7f52511c7640 1 -- 172.21.15.31:0/1053032054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240037070 con 0x7f524c107f80 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.950+0000 7f52511c7640 1 -- 172.21.15.31:0/1053032054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52400328f0 con 0x7f524c107f80 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.950+0000 7f52511c7640 1 -- 172.21.15.31:0/1053032054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240030740 con 0x7f524c107f80 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 -- 172.21.15.31:0/1053032054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 msgr2=0x7f524c10a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 --2- 172.21.15.31:0/1053032054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5240009820 tx=0x7f524002a270 comp rx=0 tx=0).stop 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 -- 172.21.15.31:0/1053032054 shutdown_connections 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 --2- 172.21.15.31:0/1053032054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c10a370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 -- 172.21.15.31:0/1053032054 >> 172.21.15.31:0/1053032054 conn(0x7f524c101b60 msgr2=0x7f524c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 -- 172.21.15.31:0/1053032054 shutdown_connections 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 -- 172.21.15.31:0/1053032054 wait complete. 2024-09-15T23:51:04.760 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.951+0000 7f525442d640 1 Processor -- start 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f525442d640 1 -- start start 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f525442d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f525442d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f524c10b8d0 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f52521c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f52521c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57670/0 (socket says 172.21.15.31:57670) 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f52521c9640 1 -- 172.21.15.31:0/4103273669 learned_addr learned my addr 172.21.15.31:0/4103273669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.952+0000 7f52521c9640 1 -- 172.21.15.31:0/4103273669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c19b970 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f52521c9640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52400097f0 tx=0x7f5240030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240042070 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5240030d80 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c19bc00 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240037040 con 0x7f524c107f80 2024-09-15T23:51:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.953+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c19e8f0 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.954+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5240045070 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.954+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c1083e0 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.954+0000 7f523f7fe640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 0x7f523803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.954+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f524006c360 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.954+0000 7f52519c8640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 0x7f523803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.955+0000 7f52519c8640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 0x7f523803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f52480098c0 tx=0x7f5248006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:03.959+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5240045520 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.138+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f524c19eac0 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.142+0000 7f523f7fe640 1 -- 172.21.15.31:0/4103273669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f524003add0 con 0x7f524c107f80 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 msgr2=0x7f523803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 0x7f523803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f52480098c0 tx=0x7f5248006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 msgr2=0x7f524c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52400097f0 tx=0x7f5240030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:04.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 shutdown_connections 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f5238039200 0x7f523803b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 --2- 172.21.15.31:0/4103273669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f524c107f80 0x7f524c19b430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 >> 172.21.15.31:0/4103273669 conn(0x7f524c101b60 msgr2=0x7f524c1026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 shutdown_connections 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:04.144+0000 7f525442d640 1 -- 172.21.15.31:0/4103273669 wait complete. 2024-09-15T23:51:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-09-15T23:51:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:04 smithi031 ceph-mon[31203]: from='mgr.14118 172.21.15.31:0/470995887' entity='mgr.x' 2024-09-15T23:51:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4103273669' entity='client.admin' 2024-09-15T23:51:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:05 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4135418187' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-15T23:51:06.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setuser ceph since I am not root 2024-09-15T23:51:06.255 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setgroup ceph since I am not root 2024-09-15T23:51:06.255 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.148+0000 7f20edbd8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:51:06.255 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.208+0000 7f20edbd8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:51:06.590 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.364+0000 7f20edbd8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:51:06.590 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.433+0000 7f20edbd8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:51:06.591 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.589+0000 7f20edbd8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:51:06.955 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:06.737+0000 7f20edbd8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:51:06.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bf4a60640 1 Processor -- start 2024-09-15T23:51:06.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bf4a60640 1 -- start start 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bf4a60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bf4a60640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf007c880 con 0x7f8bf007de50 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57676/0 (socket says 172.21.15.31:57676) 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.172+0000 7f8bee59c640 1 -- 172.21.15.31:0/2979205765 learned_addr learned my addr 172.21.15.31:0/2979205765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.173+0000 7f8bee59c640 1 -- 172.21.15.31:0/2979205765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf007ca00 con 0x7f8bf007de50 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.173+0000 7f8bee59c640 1 --2- 172.21.15.31:0/2979205765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009820 tx=0x7f8be002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda8ec14e3e0d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.173+0000 7f8bed59a640 1 -- 172.21.15.31:0/2979205765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be0037070 con 0x7f8bf007de50 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bed59a640 1 -- 172.21.15.31:0/2979205765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8be00328f0 con 0x7f8bf007de50 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bed59a640 1 -- 172.21.15.31:0/2979205765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be0030740 con 0x7f8bf007de50 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 -- 172.21.15.31:0/2979205765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 msgr2=0x7f8bf007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/2979205765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009820 tx=0x7f8be002a270 comp rx=0 tx=0).stop 2024-09-15T23:51:06.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 -- 172.21.15.31:0/2979205765 shutdown_connections 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/2979205765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf007c2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 -- 172.21.15.31:0/2979205765 >> 172.21.15.31:0/2979205765 conn(0x7f8bf0102c70 msgr2=0x7f8bf0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 -- 172.21.15.31:0/2979205765 shutdown_connections 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.174+0000 7f8bf4a60640 1 -- 172.21.15.31:0/2979205765 wait complete. 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bf4a60640 1 Processor -- start 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bf4a60640 1 -- start start 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bf4a60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bf4a60640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010db30 con 0x7f8bf007de50 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57686/0 (socket says 172.21.15.31:57686) 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.175+0000 7f8bee59c640 1 -- 172.21.15.31:0/4135418187 learned_addr learned my addr 172.21.15.31:0/4135418187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bee59c640 1 -- 172.21.15.31:0/4135418187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf019dbb0 con 0x7f8bf007de50 2024-09-15T23:51:06.965 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bee59c640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8be00097f0 tx=0x7f8be0030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be0042070 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8be0030d80 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf019de40 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be0037040 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.176+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01a0b30 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.177+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8be0045070 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.177+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf007c330 con 0x7f8bf007de50 2024-09-15T23:51:06.966 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.177+0000 7f8bdf7fe640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 0x7f8bd803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.177+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8be006c400 con 0x7f8bf007de50 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.177+0000 7f8bedd9b640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 0x7f8bd803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.178+0000 7f8bedd9b640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 0x7f8bd803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098c0 tx=0x7f8be4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.182+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8be003add0 con 0x7f8bf007de50 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.421+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8bf01a1120 con 0x7f8bf007de50 2024-09-15T23:51:06.967 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.994+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8be003d7f0 con 0x7f8bf007de50 2024-09-15T23:51:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.997+0000 7f8bdf7fe640 1 -- 172.21.15.31:0/4135418187 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f8be003c660 con 0x7f8bf007de50 2024-09-15T23:51:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.997+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 msgr2=0x7f8bd803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.997+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 0x7f8bd803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098c0 tx=0x7f8be4006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 msgr2=0x7f8bf019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8be00097f0 tx=0x7f8be0030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 shutdown_connections 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f8bd8039200 0x7f8bd803b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 --2- 172.21.15.31:0/4135418187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bf007de50 0x7f8bf019d670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 >> 172.21.15.31:0/4135418187 conn(0x7f8bf0102c70 msgr2=0x7f8bf0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 shutdown_connections 2024-09-15T23:51:06.969 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:05.998+0000 7f8bf4a60640 1 -- 172.21.15.31:0/4135418187 wait complete. 2024-09-15T23:51:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4135418187' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-15T23:51:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:06 smithi031 ceph-mon[31203]: mgrmap e9: x(active, since 19s) 2024-09-15T23:51:07.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:07.070+0000 7f20edbd8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:51:07.255 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:07.135+0000 7f20edbd8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:51:07.588 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:07.337+0000 7f20edbd8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:51:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:07 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2385661490' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T23:51:08.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a98104640 1 Processor -- start 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a98104640 1 -- start start 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a98104640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a98104640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010a630 con 0x7f3a9007bca0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a95ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a95ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57706/0 (socket says 172.21.15.31:57706) 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.380+0000 7f3a95ea0640 1 -- 172.21.15.31:0/1801053458 learned_addr learned my addr 172.21.15.31:0/1801053458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.381+0000 7f3a95ea0640 1 -- 172.21.15.31:0/1801053458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9010a7b0 con 0x7f3a9007bca0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.381+0000 7f3a95ea0640 1 --2- 172.21.15.31:0/1801053458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009820 tx=0x7f3a8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb162de10dfd57f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.381+0000 7f3a94e9e640 1 -- 172.21.15.31:0/1801053458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c037070 con 0x7f3a9007bca0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.381+0000 7f3a94e9e640 1 -- 172.21.15.31:0/1801053458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c0328f0 con 0x7f3a9007bca0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.381+0000 7f3a94e9e640 1 -- 172.21.15.31:0/1801053458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c030740 con 0x7f3a9007bca0 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 -- 172.21.15.31:0/1801053458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 msgr2=0x7f3a9010a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:08.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 --2- 172.21.15.31:0/1801053458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009820 tx=0x7f3a8c02a270 comp rx=0 tx=0).stop 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 -- 172.21.15.31:0/1801053458 shutdown_connections 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 --2- 172.21.15.31:0/1801053458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a9007bca0 0x7f3a9010a060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 -- 172.21.15.31:0/1801053458 >> 172.21.15.31:0/1801053458 conn(0x7f3a90101bb0 msgr2=0x7f3a90103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 -- 172.21.15.31:0/1801053458 shutdown_connections 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.382+0000 7f3a98104640 1 -- 172.21.15.31:0/1801053458 wait complete. 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a98104640 1 Processor -- start 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a98104640 1 -- start start 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a98104640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a98104640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010b890 con 0x7f3a90107890 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a95ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:08.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a95ea0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57720/0 (socket says 172.21.15.31:57720) 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.383+0000 7f3a95ea0640 1 -- 172.21.15.31:0/2385661490 learned_addr learned my addr 172.21.15.31:0/2385661490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a95ea0640 1 -- 172.21.15.31:0/2385661490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90107cb0 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a95ea0640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097f0 tx=0x7f3a8c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c042070 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c030d80 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a90106540 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c037040 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.384+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a90106a20 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.385+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f3a8c045070 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.385+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a90107e80 con 0x7f3a90107890 2024-09-15T23:51:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.385+0000 7f3a86ffd640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 0x7f3a7c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.385+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a8c06c380 con 0x7f3a90107890 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.385+0000 7f3a9569f640 1 -- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 msgr2=0x7f3a7c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.386+0000 7f3a9569f640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 0x7f3a7c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.390+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a8c045360 con 0x7f3a90107890 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.587+0000 7f3a9569f640 1 -- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 msgr2=0x7f3a7c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.587+0000 7f3a9569f640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 0x7f3a7c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.614+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3a9007bd40 con 0x7f3a90107890 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.614+0000 7f3a86ffd640 1 -- 172.21.15.31:0/2385661490 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3a8c02ed70 con 0x7f3a90107890 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 msgr2=0x7f3a7c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:51:08.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 0x7f3a7c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 msgr2=0x7f3a90106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097f0 tx=0x7f3a8c030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 shutdown_connections 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f3a7c039200 0x7f3a7c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 --2- 172.21.15.31:0/2385661490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a90107890 0x7f3a90106000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 >> 172.21.15.31:0/2385661490 conn(0x7f3a90101bb0 msgr2=0x7f3a90102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 shutdown_connections 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:07.616+0000 7f3a98104640 1 -- 172.21.15.31:0/2385661490 wait complete. 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-15T23:51:08.385 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-09-15T23:51:08.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:08.279+0000 7f20edbd8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:51:08.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:08.350+0000 7f20edbd8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:51:09.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.112+0000 7f20edbd8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:51:09.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.233+0000 7f20edbd8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:51:09.678 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.383+0000 7f20edbd8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:51:09.678 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.447+0000 7f20edbd8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:51:09.678 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.604+0000 7f20edbd8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:51:09.678 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.676+0000 7f20edbd8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:51:09.980 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.741+0000 7f20edbd8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:51:09.981 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.910+0000 7f20edbd8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:51:09.981 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:09.979+0000 7f20edbd8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:51:10.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:10.052+0000 7f20edbd8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:51:10.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:10.140+0000 7f20edbd8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:51:10.254 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:10.205+0000 7f20edbd8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:51:10.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:10.267+0000 7f20edbd8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:51:11.158 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:11.103+0000 7f20edbd8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:51:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: Active manager daemon x restarted 2024-09-15T23:51:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: Activating manager daemon x 2024-09-15T23:51:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: osdmap e3: 0 total, 0 up, 0 in 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: mgrmap e10: x(active, starting, since 0.00347815s) 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: Manager daemon x is now available 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T23:51:11.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T23:51:13.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.776+0000 7f9c04473640 1 Processor -- start 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c04473640 1 -- start start 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c04473640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c04473640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc07c7f0 con 0x7f9bfc07de00 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c0220f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c0220f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57722/0 (socket says 172.21.15.31:57722) 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.777+0000 7f9c0220f640 1 -- 172.21.15.31:0/1667192533 learned_addr learned my addr 172.21.15.31:0/1667192533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.778+0000 7f9c0220f640 1 -- 172.21.15.31:0/1667192533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc07c970 con 0x7f9bfc07de00 2024-09-15T23:51:13.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.778+0000 7f9c0220f640 1 --2- 172.21.15.31:0/1667192533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009820 tx=0x7f9bf802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc1f7ec4ff10846 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.778+0000 7f9c0120d640 1 -- 172.21.15.31:0/1667192533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8037070 con 0x7f9bfc07de00 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.779+0000 7f9c0120d640 1 -- 172.21.15.31:0/1667192533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf80328f0 con 0x7f9bfc07de00 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.779+0000 7f9c0120d640 1 -- 172.21.15.31:0/1667192533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf80307c0 con 0x7f9bfc07de00 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.779+0000 7f9c04473640 1 -- 172.21.15.31:0/1667192533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 msgr2=0x7f9bfc07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.779+0000 7f9c04473640 1 --2- 172.21.15.31:0/1667192533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009820 tx=0x7f9bf802a300 comp rx=0 tx=0).stop 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 -- 172.21.15.31:0/1667192533 shutdown_connections 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 --2- 172.21.15.31:0/1667192533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc07c2b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 -- 172.21.15.31:0/1667192533 >> 172.21.15.31:0/1667192533 conn(0x7f9bfc102cb0 msgr2=0x7f9bfc1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:13.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 -- 172.21.15.31:0/1667192533 shutdown_connections 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 -- 172.21.15.31:0/1667192533 wait complete. 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.780+0000 7f9c04473640 1 Processor -- start 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c04473640 1 -- start start 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c04473640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c04473640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc10dab0 con 0x7f9bfc07de00 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c0220f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c0220f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57726/0 (socket says 172.21.15.31:57726) 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c0220f640 1 -- 172.21.15.31:0/1079250867 learned_addr learned my addr 172.21.15.31:0/1079250867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c0220f640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc19db90 con 0x7f9bfc07de00 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9c0220f640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097f0 tx=0x7f9bf80307e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:13.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.781+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8037040 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.782+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf8040070 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.782+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf803bde0 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.782+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc19de20 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.782+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc1a0b10 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.782+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f9bf8045460 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.783+0000 7f9bf37fe640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.783+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9bf806c690 con 0x7f9bfc07de00 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.783+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bfc07c330 con 0x7f9be8039260 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.783+0000 7f9c01a0e640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 msgr2=0x7f9be803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.783+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.984+0000 7f9c01a0e640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 msgr2=0x7f9be803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:13.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:08.984+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:09.385+0000 7f9c01a0e640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 msgr2=0x7f9be803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:09.385+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:10.186+0000 7f9c01a0e640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 msgr2=0x7f9be803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/639707976 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:10.186+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:11.106+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f9bf806b140 con 0x7f9bfc07de00 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:11.106+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 msgr2=0x7f9be803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:11.106+0000 7f9bf37fe640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.108+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f9bf8045bc0 con 0x7f9bfc07de00 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.108+0000 7f9bf37fe640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 0x7f9be803ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.108+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bf803de70 con 0x7f9be803cb30 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.109+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 0x7f9be803ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.109+0000 7f9c01a0e640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 0x7f9be803ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9bec003980 tx=0x7f9bec0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.110+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f9bf803de70 con 0x7f9be803cb30 2024-09-15T23:51:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9bfc103f80 con 0x7f9be803cb30 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9bf37fe640 1 -- 172.21.15.31:0/1079250867 <== mgr.14150 v2:172.21.15.31:6800/797003742 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9bfc103f80 con 0x7f9be803cb30 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 msgr2=0x7f9be803ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9c04473640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 0x7f9be803ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9bec003980 tx=0x7f9bec0092b0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 msgr2=0x7f9bfc19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.117+0000 7f9c04473640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097f0 tx=0x7f9bf80307e0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 shutdown_connections 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9be803cb30 0x7f9be803ef20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:6800/639707976,v1:172.21.15.31:6801/639707976] conn(0x7f9be8039260 0x7f9be803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 --2- 172.21.15.31:0/1079250867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc07de00 0x7f9bfc19d650 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:13.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 >> 172.21.15.31:0/1079250867 conn(0x7f9bfc102cb0 msgr2=0x7f9bfc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:13.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 shutdown_connections 2024-09-15T23:51:13.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:12.118+0000 7f9c04473640 1 -- 172.21.15.31:0/1079250867 wait complete. 2024-09-15T23:51:13.036 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-09-15T23:51:13.036 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-09-15T23:51:13.297 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:13 smithi031 ceph-mon[31203]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T23:51:13.297 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:13 smithi031 ceph-mon[31203]: mgrmap e11: x(active, since 1.00585s) 2024-09-15T23:51:13.297 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:13 smithi031 ceph-mon[31203]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T23:51:13.297 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.505+0000 7f5b2b344640 1 Processor -- start 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.505+0000 7f5b2b344640 1 -- start start 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.505+0000 7f5b2b344640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.505+0000 7f5b2b344640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24107ed0 con 0x7f5b24107500 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.506+0000 7f5b290e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.506+0000 7f5b290e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50880/0 (socket says 172.21.15.31:50880) 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.506+0000 7f5b290e0640 1 -- 172.21.15.31:0/545893963 learned_addr learned my addr 172.21.15.31:0/545893963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.506+0000 7f5b290e0640 1 -- 172.21.15.31:0/545893963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b241086b0 con 0x7f5b24107500 2024-09-15T23:51:14.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.507+0000 7f5b290e0640 1 --2- 172.21.15.31:0/545893963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009820 tx=0x7f5b2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371d569d71ce5ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.507+0000 7f5b1bfff640 1 -- 172.21.15.31:0/545893963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b20037070 con 0x7f5b24107500 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.507+0000 7f5b1bfff640 1 -- 172.21.15.31:0/545893963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b200328f0 con 0x7f5b24107500 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.507+0000 7f5b1bfff640 1 -- 172.21.15.31:0/545893963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b20030740 con 0x7f5b24107500 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 -- 172.21.15.31:0/545893963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 msgr2=0x7f5b24107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 --2- 172.21.15.31:0/545893963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009820 tx=0x7f5b2002a270 comp rx=0 tx=0).stop 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 -- 172.21.15.31:0/545893963 shutdown_connections 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 --2- 172.21.15.31:0/545893963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b24107900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 -- 172.21.15.31:0/545893963 >> 172.21.15.31:0/545893963 conn(0x7f5b24102cb0 msgr2=0x7f5b241050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 -- 172.21.15.31:0/545893963 shutdown_connections 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.508+0000 7f5b2b344640 1 -- 172.21.15.31:0/545893963 wait complete. 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b2b344640 1 Processor -- start 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b2b344640 1 -- start start 2024-09-15T23:51:14.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b2b344640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b2b344640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2410ba60 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b290e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b290e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50890/0 (socket says 172.21.15.31:50890) 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b290e0640 1 -- 172.21.15.31:0/2664780946 learned_addr learned my addr 172.21.15.31:0/2664780946 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.509+0000 7f5b290e0640 1 -- 172.21.15.31:0/2664780946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2407e060 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b290e0640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b200097f0 tx=0x7f5b20030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b20042070 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b20030d80 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2407c250 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b20037040 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.510+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2407c790 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.511+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f5b20045070 con 0x7f5b24107500 2024-09-15T23:51:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.511+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b24107980 con 0x7f5b24107500 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.511+0000 7f5b1a7fc640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 0x7f5b1003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.511+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b2006c410 con 0x7f5b24107500 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.512+0000 7f5b288df640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 0x7f5b1003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.512+0000 7f5b288df640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 0x7f5b1003b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098c0 tx=0x7f5b14006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.517+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b20045520 con 0x7f5b24107500 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.706+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5b2407d110 con 0x7f5b10039260 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.752+0000 7f5b1a7fc640 1 -- 172.21.15.31:0/2664780946 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5b2407d110 con 0x7f5b10039260 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 msgr2=0x7f5b1003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 0x7f5b1003b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098c0 tx=0x7f5b14006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 msgr2=0x7f5b2407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b200097f0 tx=0x7f5b20030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:14.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 shutdown_connections 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5b10039260 0x7f5b1003b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 --2- 172.21.15.31:0/2664780946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b24107500 0x7f5b2407db20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 >> 172.21.15.31:0/2664780946 conn(0x7f5b24102cb0 msgr2=0x7f5b24103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 shutdown_connections 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:13.755+0000 7f5b2b344640 1 -- 172.21.15.31:0/2664780946 wait complete. 2024-09-15T23:51:14.355 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-09-15T23:51:14.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:14.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-mon[31203]: mgrmap e12: x(active, since 2s) 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-mon[31203]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:14.465+0000 7f2058d2c640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:23:51:14] ENGINE Error in HTTPServer.serve 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Traceback (most recent call last): 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._connections.run(self.expiration_interval) 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._run(expiration_interval) 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T23:51:14.651 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s = self.context.wrap_socket( 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: return self.sslsocket_class._create( 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.do_handshake() 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._sslobj.do_handshake() 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T23:51:14.652 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:15.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: [15/Sep/2024:23:51:14] ENGINE Bus STARTING 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: [15/Sep/2024:23:51:14] ENGINE Serving on https://172.21.15.31:7150 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: [15/Sep/2024:23:51:14] ENGINE Error in HTTPServer.serve 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T23:51:15.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: [15/Sep/2024:23:51:14] ENGINE Serving on http://172.21.15.31:8765 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: [15/Sep/2024:23:51:14] ENGINE Bus STARTED 2024-09-15T23:51:15.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:15.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:15 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:15.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jr/GPZXPV/ZDf1zQR8dZnu9OaJK1wKP5Dn/5T0hGMZKIRxkq4wAy.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726444275, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-15T23:51:15.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e8a9f4640 1 Processor -- start 2024-09-15T23:51:15.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e8a9f4640 1 -- start start 2024-09-15T23:51:15.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e8a9f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e8a9f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8407bd90 con 0x7f2e84107fb0 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50894/0 (socket says 172.21.15.31:50894) 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e83fff640 1 -- 172.21.15.31:0/2719367144 learned_addr learned my addr 172.21.15.31:0/2719367144 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.729+0000 7f2e83fff640 1 -- 172.21.15.31:0/2719367144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8410a8e0 con 0x7f2e84107fb0 2024-09-15T23:51:15.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.730+0000 7f2e83fff640 1 --2- 172.21.15.31:0/2719367144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009820 tx=0x7f2e7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ece51a43b940f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.730+0000 7f2e82ffd640 1 -- 172.21.15.31:0/2719367144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e74037070 con 0x7f2e84107fb0 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.730+0000 7f2e82ffd640 1 -- 172.21.15.31:0/2719367144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e740328f0 con 0x7f2e84107fb0 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.730+0000 7f2e82ffd640 1 -- 172.21.15.31:0/2719367144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e74030740 con 0x7f2e84107fb0 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2719367144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 msgr2=0x7f2e8410a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2719367144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009820 tx=0x7f2e7402a270 comp rx=0 tx=0).stop 2024-09-15T23:51:15.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2719367144 shutdown_connections 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2719367144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8410a3a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2719367144 >> 172.21.15.31:0/2719367144 conn(0x7f2e84101b50 msgr2=0x7f2e84103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2719367144 shutdown_connections 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2719367144 wait complete. 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.731+0000 7f2e8a9f4640 1 Processor -- start 2024-09-15T23:51:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e8a9f4640 1 -- start start 2024-09-15T23:51:15.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e8a9f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:15.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e8a9f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8410b840 con 0x7f2e84107fb0 2024-09-15T23:51:15.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:15.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50896/0 (socket says 172.21.15.31:50896) 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e83fff640 1 -- 172.21.15.31:0/2619927586 learned_addr learned my addr 172.21.15.31:0/2619927586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e83fff640 1 -- 172.21.15.31:0/2619927586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8419b970 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.732+0000 7f2e83fff640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e740097f0 tx=0x7f2e74030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e74042070 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e74030d80 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8419bc00 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e74037040 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8419e8f0 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.733+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f2e74045070 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.734+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e84108410 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.734+0000 7f2e817fa640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 0x7f2e6c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.734+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e7406c420 con 0x7f2e84107fb0 2024-09-15T23:51:15.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.734+0000 7f2e837fe640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 0x7f2e6c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.734+0000 7f2e837fe640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 0x7f2e6c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098c0 tx=0x7f2e78006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.739+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e74043260 con 0x7f2e84107fb0 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:14.928+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2e8419c0e0 con 0x7f2e6c039260 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.177+0000 7f2e817fa640 1 -- 172.21.15.31:0/2619927586 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2e8419c0e0 con 0x7f2e6c039260 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 msgr2=0x7f2e6c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 0x7f2e6c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098c0 tx=0x7f2e78006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 msgr2=0x7f2e8419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e740097f0 tx=0x7f2e74030c70 comp rx=0 tx=0).stop 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 shutdown_connections 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2e6c039260 0x7f2e6c03b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 --2- 172.21.15.31:0/2619927586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e84107fb0 0x7f2e8419b430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.179+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 >> 172.21.15.31:0/2619927586 conn(0x7f2e84101b50 msgr2=0x7f2e841026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.180+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 shutdown_connections 2024-09-15T23:51:15.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:15.180+0000 7f2e8a9f4640 1 -- 172.21.15.31:0/2619927586 wait complete. 2024-09-15T23:51:15.988 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-09-15T23:51:16.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:16 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3232866019' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe549adb640 1 Processor -- start 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe549adb640 1 -- start start 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe549adb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe549adb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe54407bd90 con 0x7fe544105de0 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50906/0 (socket says 172.21.15.31:50906) 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe5437fe640 1 -- 172.21.15.31:0/1720450039 learned_addr learned my addr 172.21.15.31:0/1720450039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.361+0000 7fe5437fe640 1 -- 172.21.15.31:0/1720450039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe544106720 con 0x7fe544105de0 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.362+0000 7fe5437fe640 1 --2- 172.21.15.31:0/1720450039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe534009820 tx=0x7fe53402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c9c76f2366576 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:17.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.362+0000 7fe5427fc640 1 -- 172.21.15.31:0/1720450039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe534037070 con 0x7fe544105de0 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.362+0000 7fe5427fc640 1 -- 172.21.15.31:0/1720450039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5340328f0 con 0x7fe544105de0 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.362+0000 7fe5427fc640 1 -- 172.21.15.31:0/1720450039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe534030740 con 0x7fe544105de0 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 -- 172.21.15.31:0/1720450039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 msgr2=0x7fe5441061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 --2- 172.21.15.31:0/1720450039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe534009820 tx=0x7fe53402a270 comp rx=0 tx=0).stop 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 -- 172.21.15.31:0/1720450039 shutdown_connections 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 --2- 172.21.15.31:0/1720450039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe5441061e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 -- 172.21.15.31:0/1720450039 >> 172.21.15.31:0/1720450039 conn(0x7fe544101bb0 msgr2=0x7fe544103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 -- 172.21.15.31:0/1720450039 shutdown_connections 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.363+0000 7fe549adb640 1 -- 172.21.15.31:0/1720450039 wait complete. 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe549adb640 1 Processor -- start 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe549adb640 1 -- start start 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe549adb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe549adb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe544107680 con 0x7fe544105de0 2024-09-15T23:51:17.496 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50920/0 (socket says 172.21.15.31:50920) 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe5437fe640 1 -- 172.21.15.31:0/3232866019 learned_addr learned my addr 172.21.15.31:0/3232866019 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.364+0000 7fe5437fe640 1 -- 172.21.15.31:0/3232866019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54419db50 con 0x7fe544105de0 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe5437fe640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe534030cc0 tx=0x7fe534030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe534042070 con 0x7fe544105de0 2024-09-15T23:51:17.497 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe534030e60 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe534037040 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54419dde0 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.365+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5441a0ad0 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.366+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fe534045070 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.366+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe544107d30 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.366+0000 7fe540ff9640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 0x7fe52c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.366+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe53406c470 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.366+0000 7fe542ffd640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 0x7fe52c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.367+0000 7fe542ffd640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 0x7fe52c03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5380098c0 tx=0x7fe538006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.371+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe534045360 con 0x7fe544105de0 2024-09-15T23:51:17.498 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.550+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe54402d9a0 con 0x7fe544105de0 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.551+0000 7fe540ff9640 1 -- 172.21.15.31:0/3232866019 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe53402ed90 con 0x7fe544105de0 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 msgr2=0x7fe52c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 0x7fe52c03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5380098c0 tx=0x7fe538006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 msgr2=0x7fe54419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe534030cc0 tx=0x7fe534030cf0 comp rx=0 tx=0).stop 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 shutdown_connections 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe52c039260 0x7fe52c03b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 --2- 172.21.15.31:0/3232866019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544105de0 0x7fe54419d610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 >> 172.21.15.31:0/3232866019 conn(0x7fe544101bb0 msgr2=0x7fe544194340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 shutdown_connections 2024-09-15T23:51:17.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:16.553+0000 7fe549adb640 1 -- 172.21.15.31:0/3232866019 wait complete. 2024-09-15T23:51:17.505 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T23:51:17.505 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-15T23:51:17.514 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T23:51:17.514 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-15T23:51:17.514 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-15T23:51:17.514 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-15T23:51:17.516 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-09-15T23:51:17.516 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:17.516 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-09-15T23:51:17.516 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-09-15T23:51:17.516 INFO:teuthology.orchestra.run.smithi031.stdout: Password: 0c8kxci90j 2024-09-15T23:51:17.517 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:17.517 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config directory 2024-09-15T23:51:18.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:18 smithi031 ceph-mon[31203]: mgrmap e13: x(active, since 6s) 2024-09-15T23:51:18.802 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:18 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/955840131' entity='client.admin' 2024-09-15T23:51:18.809 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.953+0000 7f8feccb1640 1 Processor -- start 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.953+0000 7f8feccb1640 1 -- start start 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.953+0000 7f8feccb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.953+0000 7f8feccb1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe807be20 con 0x7f8fe8107f70 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.953+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.954+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50932/0 (socket says 172.21.15.31:50932) 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.954+0000 7f8fe659c640 1 -- 172.21.15.31:0/3206320601 learned_addr learned my addr 172.21.15.31:0/3206320601 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.954+0000 7f8fe659c640 1 -- 172.21.15.31:0/3206320601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe810a8a0 con 0x7f8fe8107f70 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8fe659c640 1 --2- 172.21.15.31:0/3206320601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009820 tx=0x7f8fd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8b930d989b161e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8fe559a640 1 -- 172.21.15.31:0/3206320601 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd8037070 con 0x7f8fe8107f70 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8fe559a640 1 -- 172.21.15.31:0/3206320601 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd80328f0 con 0x7f8fe8107f70 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8fe559a640 1 -- 172.21.15.31:0/3206320601 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd8030740 con 0x7f8fe8107f70 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8feccb1640 1 -- 172.21.15.31:0/3206320601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 msgr2=0x7f8fe810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:18.810 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.955+0000 7f8feccb1640 1 --2- 172.21.15.31:0/3206320601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009820 tx=0x7f8fd802a270 comp rx=0 tx=0).stop 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 -- 172.21.15.31:0/3206320601 shutdown_connections 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 --2- 172.21.15.31:0/3206320601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe810a360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 -- 172.21.15.31:0/3206320601 >> 172.21.15.31:0/3206320601 conn(0x7f8fe8101bd0 msgr2=0x7f8fe8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 -- 172.21.15.31:0/3206320601 shutdown_connections 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 -- 172.21.15.31:0/3206320601 wait complete. 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 Processor -- start 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.956+0000 7f8feccb1640 1 -- start start 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8feccb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8feccb1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810b8c0 con 0x7f8fe8107f70 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50938/0 (socket says 172.21.15.31:50938) 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fe659c640 1 -- 172.21.15.31:0/955840131 learned_addr learned my addr 172.21.15.31:0/955840131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:18.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fe659c640 1 -- 172.21.15.31:0/955840131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe819b990 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fe659c640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8030cc0 tx=0x7f8fd8030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd8042070 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd8030e60 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe819bc20 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fd8037040 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.957+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe819e910 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.958+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8fd8045070 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.958+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe81083d0 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.958+0000 7f8fd77fe640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 0x7f8fd003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.959+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fd806c510 con 0x7f8fe8107f70 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.959+0000 7f8fe5d9b640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 0x7f8fd003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.959+0000 7f8fe5d9b640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 0x7f8fd003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0098c0 tx=0x7f8fdc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:18.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:17.963+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fd8045360 con 0x7f8fe8107f70 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.229+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8fe819eae0 con 0x7f8fe8107f70 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.231+0000 7f8fd77fe640 1 -- 172.21.15.31:0/955840131 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f8fd803ada0 con 0x7f8fe8107f70 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 msgr2=0x7f8fd003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 0x7f8fd003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0098c0 tx=0x7f8fdc006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 msgr2=0x7f8fe819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8030cc0 tx=0x7f8fd8030cf0 comp rx=0 tx=0).stop 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 shutdown_connections 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8fd0039300 0x7f8fd003b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 --2- 172.21.15.31:0/955840131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe8107f70 0x7f8fe819b450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 >> 172.21.15.31:0/955840131 conn(0x7f8fe8101bd0 msgr2=0x7f8fe81027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 shutdown_connections 2024-09-15T23:51:18.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-09-15T23:51:18.233+0000 7f8feccb1640 1 -- 172.21.15.31:0/955840131 wait complete. 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-09-15T23:51:18.814 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-09-15T23:51:18.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.815 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-15T23:51:18.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-09-15T23:51:18.851 INFO:tasks.cephadm:Fetching config... 2024-09-15T23:51:18.851 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:51:18.851 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-15T23:51:18.868 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-15T23:51:18.869 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:51:18.869 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-15T23:51:18.926 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-15T23:51:18.926 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:51:18.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-15T23:51:18.993 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-15T23:51:18.993 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:51:18.993 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-15T23:51:19.050 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-15T23:51:19.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T23:51:19.127 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:19.139 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T23:51:19.192 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:19.205 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T23:51:19.317 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjAOJXpvFkL4qJ4v/4guifIzAeCsINQ3Rw+kh3U83VzsTI2Bi7DX4c/gr1gI06qF5XEzD/bcM7oHpSegtGKdJnbIGw70WWJq0lepc2ZdvBfXjcwZBTAfEcUyxV/jElAMzjgZcKz1duE/TkThlyC/fRgJnl8vUKdWrig3l6hiQgU02LRAIS+5BtHxhotSyBNT5Xw2UjV1ScpyiaVrDxUEw6vOLPkcKwDtf7bxAxGc4Zf8SsQuzRiK1ovowIITThu4OjMsfgTqLgkoNIXqWUnCbkvmDsEp71j8dMtZeDSaeoLgbCv+ytnAGhsQVVur1AL+M7LziCegtoxr9JQwi17Nuw4v451BgFSJY8pzyqcq68u5i94uJpRL+f62blAlhhWBTe8c0bc4HcjYkBacQt5dvybWxBB1mPU8DOMIN2r8ZPcUeI5BU6Gdwl/BUii/NBoomOsbz9DqkiwSGdPfKw0ARDY24zmJ5YMh5hE18NPcArqf5dBNF0PHIPgq2qtnXmyqc= ceph-21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:19.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-15T23:51:19.586 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 -- 172.21.15.31:0/1780920508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 msgr2=0x7f80641019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 --2- 172.21.15.31:0/1780920508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f80641019e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8054009880 tx=0x7f805402a5d0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 -- 172.21.15.31:0/1780920508 shutdown_connections 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 --2- 172.21.15.31:0/1780920508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f80641019e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 -- 172.21.15.31:0/1780920508 >> 172.21.15.31:0/1780920508 conn(0x7f80640fcd70 msgr2=0x7f80640ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 -- 172.21.15.31:0/1780920508 shutdown_connections 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.036+0000 7f806982a640 1 -- 172.21.15.31:0/1780920508 wait complete. 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f806982a640 1 Processor -- start 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f806982a640 1 -- start start 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f806982a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f806982a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f80641075a0 con 0x7f80641015e0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f8062ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f8062ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50960/0 (socket says 172.21.15.31:50960) 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f8062ffd640 1 -- 172.21.15.31:0/1492563479 learned_addr learned my addr 172.21.15.31:0/1492563479 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.037+0000 7f8062ffd640 1 -- 172.21.15.31:0/1492563479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8064197240 con 0x7f80641015e0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f8062ffd640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8054005ab0 tx=0x7f80540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805403f070 con 0x7f80641015e0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8054003aa0 con 0x7f80641015e0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80641974d0 con 0x7f80641015e0 2024-09-15T23:51:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8054035030 con 0x7f80641015e0 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.038+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806419a1b0 con 0x7f80641015e0 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8054042070 con 0x7f80641015e0 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8064107d20 con 0x7f80641015e0 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f8068828640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 0x7f805003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f805406d150 con 0x7f80641015e0 2024-09-15T23:51:20.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f805bfff640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 0x7f805003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.039+0000 7f805bfff640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 0x7f805003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098c0 tx=0x7f804c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.044+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8054042360 con 0x7f80641015e0 2024-09-15T23:51:20.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.220+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f806402d940 con 0x7f80641015e0 2024-09-15T23:51:20.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.223+0000 7f8068828640 1 -- 172.21.15.31:0/1492563479 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8054003cc0 con 0x7f80641015e0 2024-09-15T23:51:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 msgr2=0x7f805003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 0x7f805003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098c0 tx=0x7f804c006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 msgr2=0x7f8064196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8054005ab0 tx=0x7f80540048f0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 shutdown_connections 2024-09-15T23:51:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f80500392a0 0x7f805003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 --2- 172.21.15.31:0/1492563479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80641015e0 0x7f8064196d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 >> 172.21.15.31:0/1492563479 conn(0x7f80640fcd70 msgr2=0x7f80640fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 shutdown_connections 2024-09-15T23:51:20.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:20.225+0000 7f806982a640 1 -- 172.21.15.31:0/1492563479 wait complete. 2024-09-15T23:51:20.810 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-15T23:51:20.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-15T23:51:21.321 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:21 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1492563479' entity='client.admin' 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 -- 172.21.15.31:0/1674941586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 msgr2=0x7fc2280733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/1674941586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc2280733f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc224009c20 tx=0x7fc22402aa90 comp rx=0 tx=0).stop 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 -- 172.21.15.31:0/1674941586 shutdown_connections 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/1674941586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc2280733f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 -- 172.21.15.31:0/1674941586 >> 172.21.15.31:0/1674941586 conn(0x7fc22806e670 msgr2=0x7fc228070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 -- 172.21.15.31:0/1674941586 shutdown_connections 2024-09-15T23:51:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.148+0000 7fc22f7aa640 1 -- 172.21.15.31:0/1674941586 wait complete. 2024-09-15T23:51:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.149+0000 7fc22f7aa640 1 Processor -- start 2024-09-15T23:51:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.149+0000 7fc22f7aa640 1 -- start start 2024-09-15T23:51:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.149+0000 7fc22f7aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.149+0000 7fc22f7aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc22810fdd0 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22d546640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22d546640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44662/0 (socket says 172.21.15.31:44662) 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22d546640 1 -- 172.21.15.31:0/459041861 learned_addr learned my addr 172.21.15.31:0/459041861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22d546640 1 -- 172.21.15.31:0/459041861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22810c2d0 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22d546640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc224006010 tx=0x7fc224030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc224047070 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc224003e20 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22810a4c0 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc224033050 con 0x7fc228072ff0 2024-09-15T23:51:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.150+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22810a9a0 con 0x7fc228072ff0 2024-09-15T23:51:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc22404a070 con 0x7fc228072ff0 2024-09-15T23:51:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228110550 con 0x7fc228072ff0 2024-09-15T23:51:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc21e7fc640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 0x7fc21403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc22403e030 con 0x7fc228072ff0 2024-09-15T23:51:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc22cd45640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 0x7fc21403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.152+0000 7fc22cd45640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 0x7fc21403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098c0 tx=0x7fc218006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.157+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2240413d0 con 0x7fc228072ff0 2024-09-15T23:51:22.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.339+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc228198850 con 0x7fc214039300 2024-09-15T23:51:22.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.342+0000 7fc21e7fc640 1 -- 172.21.15.31:0/459041861 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc228198850 con 0x7fc214039300 2024-09-15T23:51:22.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 msgr2=0x7fc21403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 0x7fc21403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098c0 tx=0x7fc218006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 msgr2=0x7fc22810bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc224006010 tx=0x7fc224030fe0 comp rx=0 tx=0).stop 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 shutdown_connections 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc214039300 0x7fc21403b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 --2- 172.21.15.31:0/459041861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc228072ff0 0x7fc22810bd90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 >> 172.21.15.31:0/459041861 conn(0x7fc22806e670 msgr2=0x7fc22806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 shutdown_connections 2024-09-15T23:51:22.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:22.345+0000 7fc22f7aa640 1 -- 172.21.15.31:0/459041861 wait complete. 2024-09-15T23:51:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:23.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:23.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:23.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:23.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:51:23.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:23.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:23.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:23 smithi031 ceph-mon[31203]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-15T23:51:23.577 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2024-09-15T23:51:23.578 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:51:23.578 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2024-09-15T23:51:23.598 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:51:23.598 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:23.658 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2024-09-15T23:51:23.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch host add smithi081 2024-09-15T23:51:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.358+0000 7f2d7d387640 1 -- 172.21.15.31:0/2215788307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 msgr2=0x7f2d78102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.358+0000 7f2d7d387640 1 --2- 172.21.15.31:0/2215788307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d78102bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009c10 tx=0x7f2d6802aa10 comp rx=0 tx=0).stop 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 -- 172.21.15.31:0/2215788307 shutdown_connections 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 --2- 172.21.15.31:0/2215788307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d78102bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 -- 172.21.15.31:0/2215788307 >> 172.21.15.31:0/2215788307 conn(0x7f2d78077310 msgr2=0x7f2d780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 -- 172.21.15.31:0/2215788307 shutdown_connections 2024-09-15T23:51:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 -- 172.21.15.31:0/2215788307 wait complete. 2024-09-15T23:51:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 Processor -- start 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.359+0000 7f2d7d387640 1 -- start start 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d7d387640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d7d387640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810bb30 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d76ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44682/0 (socket says 172.21.15.31:44682) 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d76ffd640 1 -- 172.21.15.31:0/4176948093 learned_addr learned my addr 172.21.15.31:0/4176948093 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d76ffd640 1 -- 172.21.15.31:0/4176948093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d78199800 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d76ffd640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d68005dd0 tx=0x7f2d6802f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68034070 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d6802feb0 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d78199a90 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.360+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68004360 con 0x7f2d781027d0 2024-09-15T23:51:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.361+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d78199e90 con 0x7f2d781027d0 2024-09-15T23:51:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.361+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2d68037020 con 0x7f2d781027d0 2024-09-15T23:51:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.361+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78069ef0 con 0x7f2d781027d0 2024-09-15T23:51:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.361+0000 7f2d6ffff640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 0x7f2d6403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.362+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d6806d310 con 0x7f2d781027d0 2024-09-15T23:51:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.362+0000 7f2d767fc640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 0x7f2d6403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.362+0000 7f2d767fc640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 0x7f2d6403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d600098c0 tx=0x7f2d60006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:24.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.366+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d6803f070 con 0x7f2d781027d0 2024-09-15T23:51:24.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:24.548+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}) -- 0x7f2d78108070 con 0x7f2d640392a0 2024-09-15T23:51:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:51:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T23:51:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:24 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:26.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:25 smithi031 ceph-mon[31203]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:26 smithi031 ceph-mon[31203]: Deploying cephadm binary to smithi081 2024-09-15T23:51:28.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.370+0000 7f2d6ffff640 1 -- 172.21.15.31:0/4176948093 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2d78108070 con 0x7f2d640392a0 2024-09-15T23:51:28.372 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi081' with addr '172.21.15.81' 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 msgr2=0x7f2d6403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 0x7f2d6403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d600098c0 tx=0x7f2d60006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 msgr2=0x7f2d781992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d68005dd0 tx=0x7f2d6802f980 comp rx=0 tx=0).stop 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 shutdown_connections 2024-09-15T23:51:28.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2d640392a0 0x7f2d6403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 --2- 172.21.15.31:0/4176948093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d781027d0 0x7f2d781992c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 >> 172.21.15.31:0/4176948093 conn(0x7f2d78077310 msgr2=0x7f2d78107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 shutdown_connections 2024-09-15T23:51:28.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:28.373+0000 7f2d7d387640 1 -- 172.21.15.31:0/4176948093 wait complete. 2024-09-15T23:51:29.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T23:51:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:29.623 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:29.623 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:29 smithi031 ceph-mon[31203]: Added host smithi081 2024-09-15T23:51:29.623 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:29.623 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:29.623 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- 172.21.15.31:0/3687645706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 msgr2=0x7ff4a0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/3687645706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a0102b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff490009810 tx=0x7ff49002a540 comp rx=0 tx=0).stop 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- 172.21.15.31:0/3687645706 shutdown_connections 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/3687645706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a0102b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- 172.21.15.31:0/3687645706 >> 172.21.15.31:0/3687645706 conn(0x7ff4a0077310 msgr2=0x7ff4a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- 172.21.15.31:0/3687645706 shutdown_connections 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- 172.21.15.31:0/3687645706 wait complete. 2024-09-15T23:51:29.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 Processor -- start 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.821+0000 7ff4a71cd640 1 -- start start 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a71cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a71cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010bb30 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a4f69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a4f69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44698/0 (socket says 172.21.15.31:44698) 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a4f69640 1 -- 172.21.15.31:0/1098657431 learned_addr learned my addr 172.21.15.31:0/1098657431 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a4f69640 1 -- 172.21.15.31:0/1098657431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a0199800 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a4f69640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff490030b00 tx=0x7ff490030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff490040070 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4900318b0 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a0199a90 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.822+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff490003de0 con 0x7ff4a0102730 2024-09-15T23:51:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.823+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a0199ef0 con 0x7ff4a0102730 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.823+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff490039040 con 0x7ff4a0102730 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.823+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0069f90 con 0x7ff4a0102730 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.824+0000 7ff49dffb640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 0x7ff48803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.824+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff49006d7d0 con 0x7ff4a0102730 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.824+0000 7ff49ffff640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 0x7ff48803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.824+0000 7ff49ffff640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 0x7ff48803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098c0 tx=0x7ff494006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:29.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:29.828+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff490043b60 con 0x7ff4a0102730 2024-09-15T23:51:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.009+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4a019a620 con 0x7ff488039300 2024-09-15T23:51:30.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.010+0000 7ff49dffb640 1 -- 172.21.15.31:0/1098657431 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff4a019a620 con 0x7ff488039300 2024-09-15T23:51:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:30.011 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 msgr2=0x7ff48803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 0x7ff48803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098c0 tx=0x7ff494006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 msgr2=0x7ff4a01992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff490030b00 tx=0x7ff490030b30 comp rx=0 tx=0).stop 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 shutdown_connections 2024-09-15T23:51:30.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff488039300 0x7ff48803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 --2- 172.21.15.31:0/1098657431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4a0102730 0x7ff4a01992c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 >> 172.21.15.31:0/1098657431 conn(0x7ff4a0077310 msgr2=0x7ff4a01078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 shutdown_connections 2024-09-15T23:51:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:30.012+0000 7ff4a71cd640 1 -- 172.21.15.31:0/1098657431 wait complete. 2024-09-15T23:51:30.603 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-09-15T23:51:30.603 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:51:30.603 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-09-15T23:51:30.621 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:51:30.621 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:30.677 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-09-15T23:51:30.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch host add smithi102 2024-09-15T23:51:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:30 smithi031 ceph-mon[31203]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 -- 172.21.15.31:0/842065589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 msgr2=0x7f9574102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 --2- 172.21.15.31:0/842065589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f9574102b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9570009880 tx=0x7f957002a670 comp rx=0 tx=0).stop 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 -- 172.21.15.31:0/842065589 shutdown_connections 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 --2- 172.21.15.31:0/842065589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f9574102b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 -- 172.21.15.31:0/842065589 >> 172.21.15.31:0/842065589 conn(0x7f95740fc1b0 msgr2=0x7f95740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 -- 172.21.15.31:0/842065589 shutdown_connections 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.377+0000 7f957c2e7640 1 -- 172.21.15.31:0/842065589 wait complete. 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.378+0000 7f957c2e7640 1 Processor -- start 2024-09-15T23:51:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.378+0000 7f957c2e7640 1 -- start start 2024-09-15T23:51:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957c2e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957c2e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f957410bb70 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957a083640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957a083640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44714/0 (socket says 172.21.15.31:44714) 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957a083640 1 -- 172.21.15.31:0/1901054591 learned_addr learned my addr 172.21.15.31:0/1901054591 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957a083640 1 -- 172.21.15.31:0/1901054591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741022a0 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957a083640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9570009230 tx=0x7f95700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f957003f070 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9570004d10 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741008f0 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9570035030 con 0x7f9574102750 2024-09-15T23:51:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.379+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9574100db0 con 0x7f9574102750 2024-09-15T23:51:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.380+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9570042070 con 0x7f9574102750 2024-09-15T23:51:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.380+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957406a7b0 con 0x7f9574102750 2024-09-15T23:51:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.381+0000 7f956b7fe640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 0x7f956003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.381+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f957006cfb0 con 0x7f9574102750 2024-09-15T23:51:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.381+0000 7f9579882640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 0x7f956003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.381+0000 7f9579882640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 0x7f956003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f95640098c0 tx=0x7f9564006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:31.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.385+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f957002fdc0 con 0x7f9574102750 2024-09-15T23:51:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:31.564+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) -- 0x7f95741018e0 con 0x7f95600392a0 2024-09-15T23:51:32.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:32 smithi031 ceph-mon[31203]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:32.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:32 smithi031 ceph-mon[31203]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: Deploying cephadm binary to smithi102 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:35.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:34 smithi031 ceph-mon[31203]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:35.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:34 smithi031 ceph-mon[31203]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T23:51:35.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.400+0000 7f956b7fe640 1 -- 172.21.15.31:0/1901054591 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f95741018e0 con 0x7f95600392a0 2024-09-15T23:51:35.402 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-09-15T23:51:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.402+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 msgr2=0x7f956003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.402+0000 7f957c2e7640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 0x7f956003b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f95640098c0 tx=0x7f9564006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.402+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 msgr2=0x7f95741003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.402+0000 7f957c2e7640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9570009230 tx=0x7f95700048f0 comp rx=0 tx=0).stop 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 shutdown_connections 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f95600392a0 0x7f956003b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 --2- 172.21.15.31:0/1901054591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574102750 0x7f95741003b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 >> 172.21.15.31:0/1901054591 conn(0x7f95740fc1b0 msgr2=0x7f9574105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 shutdown_connections 2024-09-15T23:51:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:35.403+0000 7f957c2e7640 1 -- 172.21.15.31:0/1901054591 wait complete. 2024-09-15T23:51:36.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T23:51:36.270 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:36.701 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:36.701 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:36.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: Added host smithi102 2024-09-15T23:51:36.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:36.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:36.702 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:36.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.699+0000 7fdfacf9f640 1 -- 172.21.15.31:0/1051585334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa8102560 msgr2=0x7fdfa8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:36.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.699+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/1051585334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa8102560 0x7fdfa8102940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009c20 tx=0x7fdf9802ab20 comp rx=0 tx=0).stop 2024-09-15T23:51:36.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.699+0000 7fdfacf9f640 1 -- 172.21.15.31:0/1051585334 shutdown_connections 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.699+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/1051585334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa8102560 0x7fdfa8102940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.699+0000 7fdfacf9f640 1 -- 172.21.15.31:0/1051585334 >> 172.21.15.31:0/1051585334 conn(0x7fdfa8077170 msgr2=0x7fdfa80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.700+0000 7fdfacf9f640 1 -- 172.21.15.31:0/1051585334 shutdown_connections 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.700+0000 7fdfacf9f640 1 -- 172.21.15.31:0/1051585334 wait complete. 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfacf9f640 1 Processor -- start 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfacf9f640 1 -- start start 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfacf9f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfacf9f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa810b980 con 0x7fdfa81001c0 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47788/0 (socket says 172.21.15.31:47788) 2024-09-15T23:51:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.701+0000 7fdfa77fe640 1 -- 172.21.15.31:0/2084478394 learned_addr learned my addr 172.21.15.31:0/2084478394 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfa77fe640 1 -- 172.21.15.31:0/2084478394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa81020b0 con 0x7fdfa81001c0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfa77fe640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005e30 tx=0x7fdf9802e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf98047070 con 0x7fdfa81001c0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf980368b0 con 0x7fdfa81001c0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa8100ae0 con 0x7fdfa81001c0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf980044b0 con 0x7fdfa81001c0 2024-09-15T23:51:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.702+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa8100fa0 con 0x7fdfa81001c0 2024-09-15T23:51:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.703+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fdf9804a070 con 0x7fdfa81001c0 2024-09-15T23:51:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.703+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa8069f90 con 0x7fdfa81001c0 2024-09-15T23:51:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.703+0000 7fdfa4ff9640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 0x7fdf9003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.703+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf9803d030 con 0x7fdfa81001c0 2024-09-15T23:51:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.704+0000 7fdfa6ffd640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 0x7fdf9003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.704+0000 7fdfa6ffd640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 0x7fdf9003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0098c0 tx=0x7fdf9c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:36.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.708+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf98032030 con 0x7fdfa81001c0 2024-09-15T23:51:36.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.888+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfa81016f0 con 0x7fdf900392a0 2024-09-15T23:51:36.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.889+0000 7fdfa4ff9640 1 -- 172.21.15.31:0/2084478394 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fdfa81016f0 con 0x7fdf900392a0 2024-09-15T23:51:36.891 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:51:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 msgr2=0x7fdf9003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 0x7fdf9003b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0098c0 tx=0x7fdf9c006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 msgr2=0x7fdfa81005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005e30 tx=0x7fdf9802e890 comp rx=0 tx=0).stop 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 shutdown_connections 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fdf900392a0 0x7fdf9003b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 --2- 172.21.15.31:0/2084478394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa81001c0 0x7fdfa81005a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 >> 172.21.15.31:0/2084478394 conn(0x7fdfa8077170 msgr2=0x7fdfa8108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 shutdown_connections 2024-09-15T23:51:36.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:36.892+0000 7fdfacf9f640 1 -- 172.21.15.31:0/2084478394 wait complete. 2024-09-15T23:51:37.541 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-15T23:51:37.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-15T23:51:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:51:37.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:37.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:37 smithi031 ceph-mon[31203]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 -- 172.21.15.31:0/1127058512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 msgr2=0x7f6d24102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1127058512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24102b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009810 tx=0x7f6d1402a540 comp rx=0 tx=0).stop 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 -- 172.21.15.31:0/1127058512 shutdown_connections 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1127058512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24102b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 -- 172.21.15.31:0/1127058512 >> 172.21.15.31:0/1127058512 conn(0x7f6d24077310 msgr2=0x7f6d240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 -- 172.21.15.31:0/1127058512 shutdown_connections 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.257+0000 7f6d2b212640 1 -- 172.21.15.31:0/1127058512 wait complete. 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d2b212640 1 Processor -- start 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d2b212640 1 -- start start 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d2b212640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d2b212640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2410bb30 con 0x7f6d24102730 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d28fae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d28fae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47804/0 (socket says 172.21.15.31:47804) 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.258+0000 7f6d28fae640 1 -- 172.21.15.31:0/1248126264 learned_addr learned my addr 172.21.15.31:0/1248126264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d28fae640 1 -- 172.21.15.31:0/1248126264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d24197580 con 0x7f6d24102730 2024-09-15T23:51:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d28fae640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d14030b00 tx=0x7f6d14030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d14040070 con 0x7f6d24102730 2024-09-15T23:51:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d24197810 con 0x7f6d24102730 2024-09-15T23:51:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d140318b0 con 0x7f6d24102730 2024-09-15T23:51:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d14003de0 con 0x7f6d24102730 2024-09-15T23:51:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.259+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d24197c70 con 0x7f6d24102730 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.260+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6d14039040 con 0x7f6d24102730 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.260+0000 7f6d21ffb640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 0x7f6d1003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.260+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d14003680 con 0x7f6d24102730 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.260+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d24069f90 con 0x7f6d24102730 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.261+0000 7f6d23fff640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 0x7f6d1003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.261+0000 7f6d23fff640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 0x7f6d1003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0098c0 tx=0x7f6d0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:38.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.265+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d140443c0 con 0x7f6d24102730 2024-09-15T23:51:38.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.446+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6d2419b910 con 0x7f6d24102730 2024-09-15T23:51:38.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.747+0000 7f6d21ffb640 1 -- 172.21.15.31:0/1248126264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6d14053030 con 0x7f6d24102730 2024-09-15T23:51:38.749 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-15T23:51:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 msgr2=0x7f6d1003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 0x7f6d1003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0098c0 tx=0x7f6d0c006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 msgr2=0x7f6d24197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d14030b00 tx=0x7f6d14030b30 comp rx=0 tx=0).stop 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 shutdown_connections 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d100392a0 0x7f6d1003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 --2- 172.21.15.31:0/1248126264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24102730 0x7f6d24197040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.749+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 >> 172.21.15.31:0/1248126264 conn(0x7f6d24077310 msgr2=0x7f6d241034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.750+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 shutdown_connections 2024-09-15T23:51:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:51:38.750+0000 7f6d2b212640 1 -- 172.21.15.31:0/1248126264 wait complete. 2024-09-15T23:51:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:38 smithi031 ceph-mon[31203]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:38 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1248126264' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T23:51:39.490 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-09-15T23:51:39.491 INFO:tasks.cephadm:Adding mon.b on smithi081 2024-09-15T23:51:39.491 INFO:tasks.cephadm:Adding mon.c on smithi102 2024-09-15T23:51:39.491 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c' 2024-09-15T23:51:39.731 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T23:51:39.775 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T23:51:40.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1248126264' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T23:51:40.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:40.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:40.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 -- 172.21.15.102:0/1437966789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 msgr2=0x7fea181019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:40.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 --2- 172.21.15.102:0/1437966789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea181019e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea08009880 tx=0x7fea0802a5d0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 -- 172.21.15.102:0/1437966789 shutdown_connections 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 --2- 172.21.15.102:0/1437966789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea181019e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 -- 172.21.15.102:0/1437966789 >> 172.21.15.102:0/1437966789 conn(0x7fea180fcd70 msgr2=0x7fea180ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 -- 172.21.15.102:0/1437966789 shutdown_connections 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.276+0000 7fea1db82640 1 -- 172.21.15.102:0/1437966789 wait complete. 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea1db82640 1 Processor -- start 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea1db82640 1 -- start start 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea1db82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea1db82640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea181075a0 con 0x7fea181015e0 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:46876/0 (socket says 172.21.15.102:46876) 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea177fe640 1 -- 172.21.15.102:0/2903503428 learned_addr learned my addr 172.21.15.102:0/2903503428 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:51:40.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.277+0000 7fea177fe640 1 -- 172.21.15.102:0/2903503428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18197240 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea177fe640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea08004ef0 tx=0x7fea080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea0803f070 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea08004c90 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea181974d0 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea08035030 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.278+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1819a1b0 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.279+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fea08042070 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.279+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea18107d20 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.279+0000 7fea14ff9640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 0x7fea0403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.279+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fea0806d0c0 con 0x7fea181015e0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.279+0000 7fea16ffd640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 0x7fea0403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.280+0000 7fea16ffd640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 0x7fea0403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea0000ad30 tx=0x7fea000093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:40.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.284+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea08042360 con 0x7fea181015e0 2024-09-15T23:51:40.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.470+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}) -- 0x7fea1802da00 con 0x7fea040392a0 2024-09-15T23:51:40.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.475+0000 7fea14ff9640 1 -- 172.21.15.102:0/2903503428 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fea1802da00 con 0x7fea040392a0 2024-09-15T23:51:40.477 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mon update... 2024-09-15T23:51:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 msgr2=0x7fea0403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 0x7fea0403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea0000ad30 tx=0x7fea000093f0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 msgr2=0x7fea18196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea08004ef0 tx=0x7fea080048f0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 shutdown_connections 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fea040392a0 0x7fea0403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.477+0000 7fea1db82640 1 --2- 172.21.15.102:0/2903503428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea181015e0 0x7fea18196d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.478+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 >> 172.21.15.102:0/2903503428 conn(0x7fea180fcd70 msgr2=0x7fea180fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.478+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 shutdown_connections 2024-09-15T23:51:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:40.478+0000 7fea1db82640 1 -- 172.21.15.102:0/2903503428 wait complete. 2024-09-15T23:51:41.060 DEBUG:teuthology.orchestra.run.smithi081:mon.b> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.b.service 2024-09-15T23:51:41.063 DEBUG:teuthology.orchestra.run.smithi102:mon.c> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.c.service 2024-09-15T23:51:41.066 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T23:51:41.067 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T23:51:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:51:41.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:41.769 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:51:42.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 -- 172.21.15.102:0/3012268464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 msgr2=0x7f8924102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:42.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 --2- 172.21.15.102:0/3012268464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924102b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8914009c10 tx=0x7f891402aa10 comp rx=0 tx=0).stop 2024-09-15T23:51:42.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 -- 172.21.15.102:0/3012268464 shutdown_connections 2024-09-15T23:51:42.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 --2- 172.21.15.102:0/3012268464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924102b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:42.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 -- 172.21.15.102:0/3012268464 >> 172.21.15.102:0/3012268464 conn(0x7f8924077310 msgr2=0x7f89240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 -- 172.21.15.102:0/3012268464 shutdown_connections 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.155+0000 7f8929921640 1 -- 172.21.15.102:0/3012268464 wait complete. 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.156+0000 7f8929921640 1 Processor -- start 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.156+0000 7f8929921640 1 -- start start 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.156+0000 7f8929921640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.156+0000 7f8929921640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f892410bb30 con 0x7f8924102790 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8922ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8922ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:46898/0 (socket says 172.21.15.102:46898) 2024-09-15T23:51:42.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8922ffd640 1 -- 172.21.15.102:0/2172913903 learned_addr learned my addr 172.21.15.102:0/2172913903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8922ffd640 1 -- 172.21.15.102:0/2172913903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8924197650 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8922ffd640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8914005dd0 tx=0x7f891402f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8914034070 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241978e0 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f891402feb0 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8914004360 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.157+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8924197ce0 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.158+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8924069f30 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.158+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8914037020 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.159+0000 7f892891f640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 0x7f890c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.159+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f891406d300 con 0x7f8924102790 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.159+0000 7f89227fc640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 0x7f890c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:42.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.159+0000 7f89227fc640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 0x7f890c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f89180098c0 tx=0x7f8918006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:42.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.164+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f891403f070 con 0x7f8924102790 2024-09-15T23:51:42.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:41 smithi031 ceph-mon[31203]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T23:51:42.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:41 smithi031 ceph-mon[31203]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:42.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:41 smithi031 ceph-mon[31203]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-15T23:51:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:41 smithi031 ceph-mon[31203]: Deploying daemon mon.c on smithi102 2024-09-15T23:51:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.397+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f892419b9e0 con 0x7f8924102790 2024-09-15T23:51:42.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.398+0000 7f892891f640 1 -- 172.21.15.102:0/2172913903 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f891402c040 con 0x7f8924102790 2024-09-15T23:51:42.399 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:51:42.400 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","modified":"2024-09-15T23:50:20.496594Z","created":"2024-09-15T23:50:20.496594Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T23:51:42.400 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 1 2024-09-15T23:51:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 msgr2=0x7f890c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 0x7f890c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f89180098c0 tx=0x7f8918006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 msgr2=0x7f8924197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:42.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8914005dd0 tx=0x7f891402f980 comp rx=0 tx=0).stop 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 shutdown_connections 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f890c0392a0 0x7f890c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 --2- 172.21.15.102:0/2172913903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8924102790 0x7f8924197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 >> 172.21.15.102:0/2172913903 conn(0x7f8924077310 msgr2=0x7f8924105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 shutdown_connections 2024-09-15T23:51:42.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:42.400+0000 7f8929921640 1 -- 172.21.15.102:0/2172913903 wait complete. 2024-09-15T23:51:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:42 smithi031 ceph-mon[31203]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:42 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/2172913903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:44.350 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T23:51:44.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T23:51:44.652 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:51:45.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3457233651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 msgr2=0x7f6a34106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:45.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3457233651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34106a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009880 tx=0x7f6a3002a5d0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3457233651 shutdown_connections 2024-09-15T23:51:45.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3457233651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34106a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3457233651 >> 172.21.15.102:0/3457233651 conn(0x7f6a340fc280 msgr2=0x7f6a340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3457233651 shutdown_connections 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.136+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3457233651 wait complete. 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.137+0000 7f6a3b7d2640 1 Processor -- start 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.137+0000 7f6a3b7d2640 1 -- start start 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.137+0000 7f6a3b7d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.137+0000 7f6a3b7d2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3410ab30 con 0x7f6a34106650 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.102:46916/0 (socket says 172.21.15.102:46916) 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3956e640 1 -- 172.21.15.102:0/3159416545 learned_addr learned my addr 172.21.15.102:0/3159416545 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3956e640 1 -- 172.21.15.102:0/3159416545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341965d0 con 0x7f6a34106650 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3956e640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009230 tx=0x7f6a300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3003f070 con 0x7f6a34106650 2024-09-15T23:51:45.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a30004d10 con 0x7f6a34106650 2024-09-15T23:51:45.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a34196860 con 0x7f6a34106650 2024-09-15T23:51:45.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.138+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a30035030 con 0x7f6a34106650 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.139+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a34196cc0 con 0x7f6a34106650 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.139+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6a30042070 con 0x7f6a34106650 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.140+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a34101cc0 con 0x7f6a34106650 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.140+0000 7f6a2a7fc640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 0x7f6a2003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.140+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a3006d2d0 con 0x7f6a34106650 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.140+0000 7f6a38d6d640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 0x7f6a2003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:45.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.141+0000 7f6a38d6d640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 0x7f6a2003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098c0 tx=0x7f6a24006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:45.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.145+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a30039350 con 0x7f6a34106650 2024-09-15T23:51:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:44 smithi031 ceph-mon[31203]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:45.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:45 smithi102 systemd[1]: Starting Ceph mon.c for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:51:45.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.398+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6a341973f0 con 0x7f6a34106650 2024-09-15T23:51:45.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.398+0000 7f6a2a7fc640 1 -- 172.21.15.102:0/3159416545 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6a3002f300 con 0x7f6a34106650 2024-09-15T23:51:45.399 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:51:45.399 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":1,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","modified":"2024-09-15T23:50:20.496594Z","created":"2024-09-15T23:50:20.496594Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T23:51:45.400 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 1 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 msgr2=0x7f6a2003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 0x7f6a2003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098c0 tx=0x7f6a24006db0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 msgr2=0x7f6a34196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009230 tx=0x7f6a300048f0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 shutdown_connections 2024-09-15T23:51:45.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6a200392a0 0x7f6a2003b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 --2- 172.21.15.102:0/3159416545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34106650 0x7f6a34196090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:45.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 >> 172.21.15.102:0/3159416545 conn(0x7f6a340fc280 msgr2=0x7f6a340fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:45.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 shutdown_connections 2024-09-15T23:51:45.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:45.400+0000 7f6a3b7d2640 1 -- 172.21.15.102:0/3159416545 wait complete. 2024-09-15T23:51:45.762 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:45 smithi102 podman[34423]: 2024-09-15 23:51:45.282172971 +0000 UTC m=+0.015439052 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:51:45.762 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:45 smithi102 podman[34423]: 2024-09-15 23:51:45.391241251 +0000 UTC m=+0.124507322 container create ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:51:46.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:45 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/3159416545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:46.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 podman[34423]: 2024-09-15 23:51:46.246321386 +0000 UTC m=+0.979587460 container init ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:51:46.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 podman[34423]: 2024-09-15 23:51:46.250330511 +0000 UTC m=+0.983596585 container start ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-15T23:51:46.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: pidfile_write: ignore empty --pid-file 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: load: jerasure load: lrc 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: RocksDB version: 7.9.2 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Git sha 0 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: DB SUMMARY 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: DB Session ID: 8TCEYPWFEQ8YV4RSRE9H 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: CURRENT file: CURRENT 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-15T23:51:46.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.error_if_exists: 0 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.create_if_missing: 0 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.paranoid_checks: 1 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.env: 0x562b6bac8940 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.info_log: 0x562b6ca056e0 2024-09-15T23:51:46.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.statistics: (nil) 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.use_fsync: 0 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_log_file_size: 0 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_fallocate: 1 2024-09-15T23:51:46.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.use_direct_reads: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.db_log_dir: 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.wal_dir: 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T23:51:46.626 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.write_buffer_manager: 0x562b6ca14dc0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.unordered_write: 0 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T23:51:46.627 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.row_cache: None 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.wal_filter: None 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.two_write_queues: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.wal_compression: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.atomic_flush: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.log_readahead_size: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T23:51:46.628 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_background_jobs: 2 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_background_compactions: -1 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_subcompactions: 1 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T23:51:46.629 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_open_files: -1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_background_flushes: -1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Compression algorithms supported: 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kZSTD supported: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kXpressCompression supported: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kBZip2Compression supported: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kLZ4Compression supported: 1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kZlibCompression supported: 1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: kSnappyCompression supported: 1 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T23:51:46.630 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.merge_operator: 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_filter: None 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562b6ca05320) 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-09-15T23:51:46.631 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_type: 0 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_index_type: 0 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_shortening: 1 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: checksum: 4 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: no_block_cache: 0 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache: 0x562b6c9f91f0 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_options: 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: capacity : 536870912 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_shard_bits : 4 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: strict_capacity_limit : 0 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_compressed: (nil) 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: persistent_cache: (nil) 2024-09-15T23:51:46.632 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size: 4096 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size_deviation: 10 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_restart_interval: 16 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_block_restart_interval: 1 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: metadata_block_size: 4096 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: partition_filters: 0 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: use_delta_encoding: 1 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: filter_policy: bloomfilter 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: whole_key_filtering: 1 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: verify_compression: 0 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: format_version: 5 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_index_compression: 1 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_align: 0 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: max_auto_readahead_size: 262144 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: prepopulate_block_cache: 0 2024-09-15T23:51:46.633 INFO:journalctl@ceph.mon.c.smithi102.stdout: initial_auto_readahead_size: 8192 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression: NoCompression 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.num_levels: 7 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T23:51:46.634 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T23:51:46.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T23:51:46.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T23:51:46.637 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.inplace_update_support: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.bloom_locality: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.max_successive_merges: 0 2024-09-15T23:51:46.638 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.ttl: 2592000 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enable_blob_files: false 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.min_blob_size: 0 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T23:51:46.639 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 21e824d3-b290-4f77-829e-68e60f032657 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444306278741, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444306279179, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726444306, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "21e824d3-b290-4f77-829e-68e60f032657", "db_session_id": "8TCEYPWFEQ8YV4RSRE9H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726444306279277, "job": 1, "event": "recovery_finished"} 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562b6ca26e00 2024-09-15T23:51:46.640 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: DB pointer 0x562b6cb22000 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** DB Stats ** 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T23:51:46.641 INFO:journalctl@ceph.mon.c.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T23:51:46.642 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache BinnedLRUCache@0x562b6c9f91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: using public_addr v2:172.21.15.102:0/0 -> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: starting mon.c rank -1 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(???) e0 preinit fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:46.643 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).mds e1 new map 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: e1 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: btime 2024-09-15T23:50:23:938450+0000 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: legacy client fscid: -1 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout: No filesystems configured 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-15T23:51:46.644 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1248126264' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1248126264' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: Deploying daemon mon.c on smithi102 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/2172913903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/3159416545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 ceph-mon[34458]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 bash[34423]: ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 2024-09-15T23:51:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:46 smithi102 systemd[1]: Started Ceph mon.c for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:51:47.170 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T23:51:47.171 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T23:51:47.427 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/2438759144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0005740 msgr2=0x7fd6e0007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/2438759144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0005740 0x7fd6e0007bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6f81016c0 tx=0x7fd6e802c3f0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/2438759144 shutdown_connections 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/2438759144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6e0005740 0x7fd6e0007bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/2438759144 >> 172.21.15.102:0/2438759144 conn(0x7fd6f80fc280 msgr2=0x7fd6f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/2438759144 shutdown_connections 2024-09-15T23:51:51.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.432+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/2438759144 wait complete. 2024-09-15T23:51:51.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.433+0000 7fd6fd8c7640 1 Processor -- start 2024-09-15T23:51:51.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.433+0000 7fd6fd8c7640 1 -- start start 2024-09-15T23:51:51.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.433+0000 7fd6fd8c7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:51.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6fd8c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 0x7fd6f8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:51.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6fd8c7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8107570 con 0x7fd6f8195490 2024-09-15T23:51:51.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6fd8c7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81073f0 con 0x7fd6e0005740 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43440/0 (socket says 172.21.15.102:43440) 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 -- 172.21.15.102:0/1007097210 learned_addr learned my addr 172.21.15.102:0/1007097210 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 -- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 msgr2=0x7fd6f8194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 -- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 msgr2=0x7fd6f8194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f6ffd640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f67fc640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 0x7fd6f8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f67fc640 1 -- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 msgr2=0x7fd6f8194f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f67fc640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f67fc640 1 -- 172.21.15.102:0/1007097210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f8195ab0 con 0x7fd6f8195490 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.434+0000 7fd6f67fc640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 0x7fd6f8199820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00b3d0 tx=0x7fd6ec00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.435+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd6ec011630 con 0x7fd6f8195490 2024-09-15T23:51:51.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.435+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6ec011c80 con 0x7fd6f8195490 2024-09-15T23:51:51.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.435+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd6ec010d80 con 0x7fd6f8195490 2024-09-15T23:51:51.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.435+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f8199d60 con 0x7fd6f8195490 2024-09-15T23:51:51.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.435+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f819a320 con 0x7fd6f8195490 2024-09-15T23:51:51.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.436+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd6ec018020 con 0x7fd6f8195490 2024-09-15T23:51:51.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.436+0000 7fd6fc8c5640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 0x7fd6e403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:51.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.436+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd6ec010510 con 0x7fd6f8195490 2024-09-15T23:51:51.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.436+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f819a630 con 0x7fd6f8195490 2024-09-15T23:51:51.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.436+0000 7fd6f6ffd640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 0x7fd6e403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:51.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.437+0000 7fd6f6ffd640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 0x7fd6e403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8014040 tx=0x7fd6e80149c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:51.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.441+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6ec01f020 con 0x7fd6f8195490 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: mon.a calling monitor election 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: mon.c calling monitor election 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: monmap epoch 2 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: last_changed 2024-09-15T23:51:46.305106+0000 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: min_mon_release 19 (squid) 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: election_strategy: 1 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: fsmap 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: mgrmap e13: x(active, since 40s) 2024-09-15T23:51:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: overall HEALTH_OK 2024-09-15T23:51:51.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:51:51.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:51.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.682+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd6f81962e0 con 0x7fd6f8195490 2024-09-15T23:51:51.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.683+0000 7fd6fc8c5640 1 -- 172.21.15.102:0/1007097210 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd6ec014170 con 0x7fd6f8195490 2024-09-15T23:51:51.684 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:51:51.684 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","modified":"2024-09-15T23:51:46.305106Z","created":"2024-09-15T23:50:20.496594Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T23:51:51.684 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-09-15T23:51:51.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 msgr2=0x7fd6e403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:51.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 0x7fd6e403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8014040 tx=0x7fd6e80149c0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 msgr2=0x7fd6f8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 0x7fd6f8199820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00b3d0 tx=0x7fd6ec00b890 comp rx=0 tx=0).stop 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 shutdown_connections 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd6e4039090 0x7fd6e403b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f8195490 0x7fd6f8199820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 --2- 172.21.15.102:0/1007097210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6e0005740 0x7fd6f8194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.685+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 >> 172.21.15.102:0/1007097210 conn(0x7fd6f80fc280 msgr2=0x7fd6f81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.686+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 shutdown_connections 2024-09-15T23:51:51.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:51.686+0000 7fd6fd8c7640 1 -- 172.21.15.102:0/1007097210 wait complete. 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: mon.a calling monitor election 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: mon.c calling monitor election 2024-09-15T23:51:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: monmap epoch 2 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: last_changed 2024-09-15T23:51:46.305106+0000 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: min_mon_release 19 (squid) 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: election_strategy: 1 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: fsmap 2024-09-15T23:51:51.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: mgrmap e13: x(active, since 40s) 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: overall HEALTH_OK 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:51:51.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:51:52.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:52 smithi102 ceph-mon[34458]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:52.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:52 smithi102 ceph-mon[34458]: Deploying daemon mon.b on smithi081 2024-09-15T23:51:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:52 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1007097210' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:52.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:52 smithi031 ceph-mon[31203]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:52.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:52 smithi031 ceph-mon[31203]: Deploying daemon mon.b on smithi081 2024-09-15T23:51:52.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:52 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1007097210' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:52.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:51:53.289 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T23:51:53.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T23:51:53.548 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:51:53.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:51:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:51:53.304+0000 7f208c8e1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T23:51:53.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 -- 172.21.15.102:0/4082876720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28106630 msgr2=0x7f7b28106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:53.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 --2- 172.21.15.102:0/4082876720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28106630 0x7f7b28106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009880 tx=0x7f7b1c02a5d0 comp rx=0 tx=0).stop 2024-09-15T23:51:53.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 -- 172.21.15.102:0/4082876720 shutdown_connections 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 --2- 172.21.15.102:0/4082876720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28106630 0x7f7b28106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 -- 172.21.15.102:0/4082876720 >> 172.21.15.102:0/4082876720 conn(0x7f7b280fc220 msgr2=0x7f7b280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 -- 172.21.15.102:0/4082876720 shutdown_connections 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.959+0000 7f7b2f005640 1 -- 172.21.15.102:0/4082876720 wait complete. 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 Processor -- start 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 -- start start 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 0x7f7b2819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2810ab10 con 0x7f7b28196630 2024-09-15T23:51:53.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2f005640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2810a990 con 0x7f7b28106630 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2cda1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2cda1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43450/0 (socket says 172.21.15.102:43450) 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2cda1640 1 -- 172.21.15.102:0/352446375 learned_addr learned my addr 172.21.15.102:0/352446375 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2cda1640 1 -- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 msgr2=0x7f7b281960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.960+0000 7f7b2cda1640 1 -- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 msgr2=0x7f7b281960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b2cda1640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b2cda1640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b27fff640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 0x7f7b2819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b27fff640 1 -- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 msgr2=0x7f7b281960f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:51:53.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b27fff640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b27fff640 1 -- 172.21.15.102:0/352446375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b28196c50 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b27fff640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 0x7f7b2819a9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b3d0 tx=0x7f7b1400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7b14011630 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b14011c80 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7b14010d80 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.961+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2819af00 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.962+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2819b4c0 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.962+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7b14018020 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.962+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2819b7d0 con 0x7f7b28196630 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.963+0000 7f7b25ffb640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 0x7f7b1803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:51:53.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.963+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b1404bfe0 con 0x7f7b28196630 2024-09-15T23:51:53.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.963+0000 7f7b2cda1640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 0x7f7b1803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:51:53.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.963+0000 7f7b2cda1640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 0x7f7b1803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0314f0 tx=0x7f7b1c031420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:51:53.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:53.968+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b1401f020 con 0x7f7b28196630 2024-09-15T23:51:54.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.197+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7b28197450 con 0x7f7b28196630 2024-09-15T23:51:54.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.198+0000 7f7b25ffb640 1 -- 172.21.15.102:0/352446375 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7b14017dd0 con 0x7f7b28196630 2024-09-15T23:51:54.199 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:51:54.199 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","modified":"2024-09-15T23:51:46.305106Z","created":"2024-09-15T23:50:20.496594Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T23:51:54.199 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 msgr2=0x7f7b1803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 0x7f7b1803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0314f0 tx=0x7f7b1c031420 comp rx=0 tx=0).stop 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 msgr2=0x7f7b2819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 0x7f7b2819a9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b3d0 tx=0x7f7b1400b890 comp rx=0 tx=0).stop 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 shutdown_connections 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7b18039040 0x7f7b1803b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b28196630 0x7f7b2819a9c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 --2- 172.21.15.102:0/352446375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b28106630 0x7f7b281960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:51:54.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 >> 172.21.15.102:0/352446375 conn(0x7f7b280fc220 msgr2=0x7f7b280fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:51:54.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 shutdown_connections 2024-09-15T23:51:54.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:51:54.200+0000 7f7b2f005640 1 -- 172.21.15.102:0/352446375 wait complete. 2024-09-15T23:51:54.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:54 smithi102 ceph-mon[34458]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:54.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:51:54 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/352446375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:54.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:54 smithi031 ceph-mon[31203]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:51:54.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:51:54 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/352446375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:51:55.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:51:55 smithi081 ceph-mon[35060]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T23:51:55.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:51:55 smithi081 bash[35042]: 94c27af25f7a12143b63a1c2a63976b3067ac767870305bc4c315a98c8d9b804 2024-09-15T23:51:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:51:55 smithi081 systemd[1]: Started Ceph mon.b for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:51:55.729 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T23:51:55.729 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T23:51:55.982 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: mon.a calling monitor election 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: mon.c calling monitor election 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: monmap epoch 3 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: min_mon_release 19 (squid) 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: election_strategy: 1 2024-09-15T23:52:00.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: fsmap 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: mgrmap e13: x(active, since 48s) 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: overall HEALTH_OK 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: mon.a calling monitor election 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: mon.c calling monitor election 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: monmap epoch 3 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: min_mon_release 19 (squid) 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: election_strategy: 1 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: fsmap 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: mgrmap e13: x(active, since 48s) 2024-09-15T23:52:00.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: overall HEALTH_OK 2024-09-15T23:52:00.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:00.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.802+0000 7fa8d0846640 1 -- 172.21.15.102:0/1616513160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b8005740 msgr2=0x7fa8b8007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:00.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.802+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1616513160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b8005740 0x7fa8b8007bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc1016a0 tx=0x7fa8c402c490 comp rx=0 tx=0).stop 2024-09-15T23:52:00.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 -- 172.21.15.102:0/1616513160 shutdown_connections 2024-09-15T23:52:00.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1616513160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b8005740 0x7fa8b8007bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:00.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 -- 172.21.15.102:0/1616513160 >> 172.21.15.102:0/1616513160 conn(0x7fa8cc0fc220 msgr2=0x7fa8cc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:00.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 -- 172.21.15.102:0/1616513160 shutdown_connections 2024-09-15T23:52:00.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 -- 172.21.15.102:0/1616513160 wait complete. 2024-09-15T23:52:00.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.803+0000 7fa8d0846640 1 Processor -- start 2024-09-15T23:52:00.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 -- start start 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 0x7fa8cc19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc107550 con 0x7fa8cc1999e0 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1073d0 con 0x7fa8b8005740 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8d0846640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1076d0 con 0x7fa8cc19e2b0 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59984/0 (socket says 172.21.15.102:59984) 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 -- 172.21.15.102:0/1200962158 learned_addr learned my addr 172.21.15.102:0/1200962158 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 msgr2=0x7fa8cc19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T23:52:00.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 msgr2=0x7fa8cc19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8cad9d640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8c9d9b640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 0x7fa8cc19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8ca59c640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8ca59c640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 msgr2=0x7fa8cc1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8ca59c640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 msgr2=0x7fa8cc1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8ca59c640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.804+0000 7fa8ca59c640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 msgr2=0x7fa8cc19e760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 msgr2=0x7fa8cc1994a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:00.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 -- 172.21.15.102:0/1200962158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc1a6020 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c9d9b640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 0x7fa8cc19dd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400b450 tx=0x7fa8b400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4004230 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8b40043d0 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1a6250 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4003be0 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.805+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a67e0 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.806+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc1084e0 con 0x7fa8cc1999e0 2024-09-15T23:52:00.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.806+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa8b400f050 con 0x7fa8cc1999e0 2024-09-15T23:52:00.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.807+0000 7fa8c37fe640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 0x7fa8ac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:00.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.807+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8b404c080 con 0x7fa8cc1999e0 2024-09-15T23:52:00.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.807+0000 7fa8ca59c640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 0x7fa8ac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:00.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.807+0000 7fa8ca59c640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 0x7fa8ac03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005940 tx=0x7fa8c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:00.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:00.812+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8b400d040 con 0x7fa8cc1999e0 2024-09-15T23:52:01.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.045+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa8cc02d7c0 con 0x7fa8cc1999e0 2024-09-15T23:52:01.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.045+0000 7fa8c37fe640 1 -- 172.21.15.102:0/1200962158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fa8b4021db0 con 0x7fa8cc1999e0 2024-09-15T23:52:01.046 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:52:01.047 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","modified":"2024-09-15T23:51:55.025989Z","created":"2024-09-15T23:50:20.496594Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T23:52:01.047 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 3 2024-09-15T23:52:01.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 msgr2=0x7fa8ac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:01.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 0x7fa8ac03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005940 tx=0x7fa8c4005850 comp rx=0 tx=0).stop 2024-09-15T23:52:01.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 msgr2=0x7fa8cc19dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:01.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 0x7fa8cc19dd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400b450 tx=0x7fa8b400b910 comp rx=0 tx=0).stop 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 shutdown_connections 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa8ac039730 0x7fa8ac03bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8cc19e2b0 0x7fa8cc19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8cc1999e0 0x7fa8cc19dd70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 --2- 172.21.15.102:0/1200962158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b8005740 0x7fa8cc1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.047+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 >> 172.21.15.102:0/1200962158 conn(0x7fa8cc0fc220 msgr2=0x7fa8cc1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.048+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 shutdown_connections 2024-09-15T23:52:01.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:01.048+0000 7fa8d0846640 1 -- 172.21.15.102:0/1200962158 wait complete. 2024-09-15T23:52:01.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:01.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:01.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:01 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1200962158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:52:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:01 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1200962158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:52:01.888 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-15T23:52:01.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-15T23:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.a calling monitor election 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.c calling monitor election 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: monmap epoch 3 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:02.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: min_mon_release 19 (squid) 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: election_strategy: 1 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: fsmap 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mgrmap e13: x(active, since 48s) 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: overall HEALTH_OK 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:02.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:02.310 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/1200962158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T23:52:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 -- 172.21.15.31:0/2443715570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60107660 msgr2=0x7f9f60107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/2443715570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60107660 0x7f9f60107a40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009880 tx=0x7f9f5002a5d0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 -- 172.21.15.31:0/2443715570 shutdown_connections 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/2443715570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60107660 0x7f9f60107a40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 -- 172.21.15.31:0/2443715570 >> 172.21.15.31:0/2443715570 conn(0x7f9f600fd290 msgr2=0x7f9f600ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 -- 172.21.15.31:0/2443715570 shutdown_connections 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.577+0000 7f9f65cbb640 1 -- 172.21.15.31:0/2443715570 wait complete. 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 Processor -- start 2024-09-15T23:52:02.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 -- start start 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f60107660 0x7f9f60197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 0x7f9f6019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6010baf0 con 0x7f9f60197870 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6010b970 con 0x7f9f6019c140 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f65cbb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6010bc70 con 0x7f9f60107660 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f5effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35848/0 (socket says 172.21.15.31:35848) 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.578+0000 7f9f5effd640 1 -- 172.21.15.31:0/620170612 learned_addr learned my addr 172.21.15.31:0/620170612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5f7fe640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f60107660 0x7f9f60197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5ffff640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 0x7f9f6019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:02.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 -- 172.21.15.31:0/620170612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f60107660 msgr2=0x7f9f60197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:02.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f60107660 0x7f9f60197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 -- 172.21.15.31:0/620170612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 msgr2=0x7f9f6019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 0x7f9f6019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 -- 172.21.15.31:0/620170612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f601a3eb0 con 0x7f9f60197870 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5ffff640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 0x7f9f6019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5effd640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400b520 tx=0x7f9f5400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54004230 con 0x7f9f60197870 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f601a40e0 con 0x7f9f60197870 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f540043d0 con 0x7f9f60197870 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003cf0 con 0x7f9f60197870 2024-09-15T23:52:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.579+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f601a4640 con 0x7f9f60197870 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.580+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f60107ac0 con 0x7f9f60197870 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.581+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9f5400e070 con 0x7f9f60197870 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.581+0000 7f9f5cff9640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 0x7f9f4803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.581+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f5404bd50 con 0x7f9f60197870 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.581+0000 7f9f5f7fe640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 0x7f9f4803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.581+0000 7f9f5f7fe640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 0x7f9f4803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f50031440 tx=0x7f9f50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.586+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f5404f030 con 0x7f9f60197870 2024-09-15T23:52:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.764+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9f601a4a90 con 0x7f9f60197870 2024-09-15T23:52:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.765+0000 7f9f5cff9640 1 -- 172.21.15.31:0/620170612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f9f54014020 con 0x7f9f60197870 2024-09-15T23:52:02.766 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:02.766 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-15T23:52:02.766 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:02.766 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-09-15T23:52:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 msgr2=0x7f9f4803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 0x7f9f4803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f50031440 tx=0x7f9f50005a20 comp rx=0 tx=0).stop 2024-09-15T23:52:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 msgr2=0x7f9f6019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400b520 tx=0x7f9f5400b9e0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 shutdown_connections 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9f48039730 0x7f9f4803bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f6019c140 0x7f9f6019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f60197870 0x7f9f6019bc00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 --2- 172.21.15.31:0/620170612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f60107660 0x7f9f60197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 >> 172.21.15.31:0/620170612 conn(0x7f9f600fd290 msgr2=0x7f9f600fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 shutdown_connections 2024-09-15T23:52:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:02.767+0000 7f9f65cbb640 1 -- 172.21.15.31:0/620170612 wait complete. 2024-09-15T23:52:02.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.b calling monitor election 2024-09-15T23:52:02.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:02.951 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: Reconfiguring daemon mon.a on smithi031 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.b calling monitor election 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.c calling monitor election 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.a calling monitor election 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: monmap epoch 3 2024-09-15T23:52:02.952 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: min_mon_release 19 (squid) 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: election_strategy: 1 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: fsmap 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: mgrmap e13: x(active, since 50s) 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: overall HEALTH_OK 2024-09-15T23:52:02.953 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:02 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/620170612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mon.b calling monitor election 2024-09-15T23:52:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-15T23:52:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-15T23:52:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: Reconfiguring daemon mon.a on smithi031 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mon.b calling monitor election 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mon.c calling monitor election 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mon.a calling monitor election 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: monmap epoch 3 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:03.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: min_mon_release 19 (squid) 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: election_strategy: 1 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: fsmap 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: mgrmap e13: x(active, since 50s) 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: overall HEALTH_OK 2024-09-15T23:52:03.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:02 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/620170612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mon.b calling monitor election 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T23:52:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: Reconfiguring daemon mon.a on smithi031 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mon.b calling monitor election 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mon.c calling monitor election 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mon.a calling monitor election 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: monmap epoch 3 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: last_changed 2024-09-15T23:51:55.025989+0000 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: created 2024-09-15T23:50:20.496594+0000 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: min_mon_release 19 (squid) 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: election_strategy: 1 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: 1: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.c 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-09-15T23:52:03.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: fsmap 2024-09-15T23:52:03.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T23:52:03.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: mgrmap e13: x(active, since 50s) 2024-09-15T23:52:03.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: overall HEALTH_OK 2024-09-15T23:52:03.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:02 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/620170612' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:03.780 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-15T23:52:03.780 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:52:03.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T23:52:03.808 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:52:03.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:52:03.874 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:52:03.875 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T23:52:03.908 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:52:03.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:52:03.981 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:52:03.981 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T23:52:04.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:03 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:04.013 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:52:04.013 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T23:52:04.047 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-09-15T23:52:04.047 INFO:tasks.cephadm:Adding mgr.y on smithi081 2024-09-15T23:52:04.047 INFO:tasks.cephadm:Adding mgr.z on smithi102 2024-09-15T23:52:04.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi031=x;smithi081=y;smithi102=z' 2024-09-15T23:52:04.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:03 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:04.334 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:52:04.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:03 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 -- 172.21.15.102:0/3711458728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c1016e0 msgr2=0x7f545c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 --2- 172.21.15.102:0/3711458728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c1016e0 0x7f545c101ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f544c009880 tx=0x7f544c02a5d0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 -- 172.21.15.102:0/3711458728 shutdown_connections 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 --2- 172.21.15.102:0/3711458728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c1016e0 0x7f545c101ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 -- 172.21.15.102:0/3711458728 >> 172.21.15.102:0/3711458728 conn(0x7f545c0fce90 msgr2=0x7f545c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 -- 172.21.15.102:0/3711458728 shutdown_connections 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 -- 172.21.15.102:0/3711458728 wait complete. 2024-09-15T23:52:04.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.753+0000 7f5461f34640 1 Processor -- start 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 -- start start 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 0x7f545c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c197470 0x7f545c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1076a0 con 0x7f545c197470 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f545c107520 con 0x7f545c1016e0 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f5461f34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f545c107820 con 0x7f545c1978f0 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60000/0 (socket says 172.21.15.102:60000) 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545bfff640 1 -- 172.21.15.102:0/1968693457 learned_addr learned my addr 172.21.15.102:0/1968693457 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 0x7f545c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545affd640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c197470 0x7f545c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:04.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 -- 172.21.15.102:0/1968693457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 msgr2=0x7f545c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 -- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c197470 msgr2=0x7f545c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c197470 0x7f545c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545b7fe640 1 -- 172.21.15.102:0/1968693457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19f330 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.754+0000 7f545bfff640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f545b7fe640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 0x7f545c196f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f544c006010 tx=0x7f544c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c03f070 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544c003e40 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19f5c0 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c035030 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.755+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c19faf0 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.756+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f544c042070 con 0x7f545c1016e0 2024-09-15T23:52:04.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.756+0000 7f5458ff9640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 0x7f544403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:04.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.756+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c190460 con 0x7f545c1016e0 2024-09-15T23:52:04.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.756+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f544c06d010 con 0x7f545c1016e0 2024-09-15T23:52:04.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.756+0000 7f545affd640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 0x7f544403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:04.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.757+0000 7f545affd640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 0x7f544403bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f545c198450 tx=0x7f5450009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:04.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.761+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f544c03bdf0 con 0x7f545c1016e0 2024-09-15T23:52:04.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.938+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}) -- 0x7f545c0626c0 con 0x7f5444039730 2024-09-15T23:52:04.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.944+0000 7f5458ff9640 1 -- 172.21.15.102:0/1968693457 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f545c0626c0 con 0x7f5444039730 2024-09-15T23:52:04.945 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mgr update... 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 msgr2=0x7f544403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 0x7f544403bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f545c198450 tx=0x7f5450009190 comp rx=0 tx=0).stop 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 msgr2=0x7f545c196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 0x7f545c196f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f544c006010 tx=0x7f544c003cd0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 shutdown_connections 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5444039730 0x7f544403bbf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f545c1978f0 0x7f545c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f545c197470 0x7f545c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 --2- 172.21.15.102:0/1968693457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f545c1016e0 0x7f545c196f30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:04.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 >> 172.21.15.102:0/1968693457 conn(0x7f545c0fce90 msgr2=0x7f545c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:04.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 shutdown_connections 2024-09-15T23:52:04.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:52:04.946+0000 7f5461f34640 1 -- 172.21.15.102:0/1968693457 wait complete. 2024-09-15T23:52:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:05.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:05.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:05.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:05.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.268 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:05.268 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.268 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.268 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:05.268 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:05.269 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:05.269 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:05.488 DEBUG:teuthology.orchestra.run.smithi081:mgr.y> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y.service 2024-09-15T23:52:05.491 DEBUG:teuthology.orchestra.run.smithi102:mgr.z> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z.service 2024-09-15T23:52:05.495 INFO:tasks.cephadm:Deploying OSDs... 2024-09-15T23:52:05.495 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:52:05.495 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T23:52:05.518 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T23:52:05.518 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:05.577 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-15 23:51:22.093164852 +0000 2024-09-15T23:52:05.578 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-15 23:47:19.281269964 +0000 2024-09-15T23:52:05.578 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-15 23:47:19.281269964 +0000 2024-09-15T23:52:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-15 23:47:19.281269964 +0000 2024-09-15T23:52:05.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T23:52:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-15T23:52:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-15T23:52:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000285335 s, 1.8 MB/s 2024-09-15T23:52:05.648 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T23:52:05.707 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-15 23:51:22.128164280 +0000 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-15 23:47:19.616264335 +0000 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-15 23:47:19.616264335 +0000 2024-09-15T23:52:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-15 23:47:19.616264335 +0000 2024-09-15T23:52:05.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T23:52:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-15T23:52:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-15T23:52:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000229631 s, 2.2 MB/s 2024-09-15T23:52:05.834 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T23:52:05.892 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-15 23:51:22.167163644 +0000 2024-09-15T23:52:05.950 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-15 23:47:19.969258403 +0000 2024-09-15T23:52:05.951 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-15 23:47:19.969258403 +0000 2024-09-15T23:52:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-15 23:47:19.969258403 +0000 2024-09-15T23:52:05.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T23:52:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-15T23:52:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-15T23:52:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000201068 s, 2.5 MB/s 2024-09-15T23:52:06.018 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T23:52:06.078 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-15T23:52:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-15 23:51:22.202163073 +0000 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-15 23:47:20.315252589 +0000 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-15 23:47:20.315252589 +0000 2024-09-15T23:52:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-15 23:47:20.315252589 +0000 2024-09-15T23:52:06.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T23:52:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-15T23:52:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-15T23:52:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000249554 s, 2.1 MB/s 2024-09-15T23:52:06.202 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T23:52:06.238 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: Reconfiguring mon.b (monmap changed)... 2024-09-15T23:52:06.238 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: Reconfiguring daemon mon.b on smithi081 2024-09-15T23:52:06.238 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:06.239 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: Reconfiguring mon.b (monmap changed)... 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: Reconfiguring daemon mon.b on smithi081 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:06.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:06.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:06.260 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-15T23:52:06.260 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T23:52:06.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T23:52:06.282 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: Reconfiguring mon.b (monmap changed)... 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: Reconfiguring daemon mon.b on smithi081 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi081=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: Saving service mgr spec with placement smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T23:52:06.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T23:52:06.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:06.323 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T23:52:06.323 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:06.323 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T23:52:06.323 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:06.324 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:06.324 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-15 23:51:54.510357205 +0000 2024-09-15T23:52:06.324 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-15 23:47:17.916658174 +0000 2024-09-15T23:52:06.324 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-15 23:47:17.916658174 +0000 2024-09-15T23:52:06.324 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-15 23:47:17.916658174 +0000 2024-09-15T23:52:06.324 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T23:52:06.388 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-15T23:52:06.388 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-15T23:52:06.389 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000223978 s, 2.3 MB/s 2024-09-15T23:52:06.390 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T23:52:06.447 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-15 23:51:54.510357205 +0000 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-15 23:47:18.241652130 +0000 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-15 23:47:18.241652130 +0000 2024-09-15T23:52:06.503 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-15 23:47:18.241652130 +0000 2024-09-15T23:52:06.504 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T23:52:06.566 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-15T23:52:06.566 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-15T23:52:06.566 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000195366 s, 2.6 MB/s 2024-09-15T23:52:06.567 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T23:52:06.624 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-09-15T23:52:06.680 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T23:52:06.680 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:06.680 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T23:52:06.680 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:06.680 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:06.681 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-15 23:51:54.511357187 +0000 2024-09-15T23:52:06.681 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-15 23:47:18.564646123 +0000 2024-09-15T23:52:06.681 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-15 23:47:18.564646123 +0000 2024-09-15T23:52:06.681 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-15 23:47:18.564646123 +0000 2024-09-15T23:52:06.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T23:52:06.743 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-15T23:52:06.744 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-15T23:52:06.744 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000181353 s, 2.8 MB/s 2024-09-15T23:52:06.745 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T23:52:06.801 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-09-15T23:52:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T23:52:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:06.857 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T23:52:06.857 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:06.858 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:06.858 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-15 23:51:54.511357187 +0000 2024-09-15T23:52:06.858 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-15 23:47:18.888640098 +0000 2024-09-15T23:52:06.858 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-15 23:47:18.888640098 +0000 2024-09-15T23:52:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-15 23:47:18.888640098 +0000 2024-09-15T23:52:06.858 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T23:52:06.921 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-15T23:52:06.921 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-15T23:52:06.921 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000169173 s, 3.0 MB/s 2024-09-15T23:52:06.922 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T23:52:06.977 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-15T23:52:06.977 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T23:52:06.996 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T23:52:06.996 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-15T23:52:07.052 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-15 23:51:45.083688975 +0000 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-15 23:47:19.350995767 +0000 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-15 23:47:19.350995767 +0000 2024-09-15T23:52:07.053 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-15 23:47:19.350995767 +0000 2024-09-15T23:52:07.053 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T23:52:07.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:06 smithi081 ceph-mon[35060]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:07.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:06 smithi081 ceph-mon[35060]: Reconfiguring mon.c (monmap changed)... 2024-09-15T23:52:07.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:06 smithi081 ceph-mon[35060]: Reconfiguring daemon mon.c on smithi102 2024-09-15T23:52:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-15T23:52:07.116 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-15T23:52:07.117 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000256046 s, 2.0 MB/s 2024-09-15T23:52:07.118 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T23:52:07.175 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-15T23:52:07.229 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-15 23:51:45.084688956 +0000 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-15 23:47:19.692988891 +0000 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-15 23:47:19.692988891 +0000 2024-09-15T23:52:07.230 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-15 23:47:19.692988891 +0000 2024-09-15T23:52:07.230 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T23:52:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:06 smithi031 ceph-mon[31203]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:06 smithi031 ceph-mon[31203]: Reconfiguring mon.c (monmap changed)... 2024-09-15T23:52:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:06 smithi031 ceph-mon[31203]: Reconfiguring daemon mon.c on smithi102 2024-09-15T23:52:07.293 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:06 smithi102 ceph-mon[34458]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:07.293 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:06 smithi102 ceph-mon[34458]: Reconfiguring mon.c (monmap changed)... 2024-09-15T23:52:07.293 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:06 smithi102 ceph-mon[34458]: Reconfiguring daemon mon.c on smithi102 2024-09-15T23:52:07.294 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-15T23:52:07.294 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-15T23:52:07.294 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000207484 s, 2.5 MB/s 2024-09-15T23:52:07.296 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T23:52:07.353 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-15 23:51:45.084688956 +0000 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-15 23:47:20.019982316 +0000 2024-09-15T23:52:07.409 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-15 23:47:20.019982316 +0000 2024-09-15T23:52:07.410 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-15 23:47:20.019982316 +0000 2024-09-15T23:52:07.410 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T23:52:07.472 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-15T23:52:07.473 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-15T23:52:07.473 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000226504 s, 2.3 MB/s 2024-09-15T23:52:07.474 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T23:52:07.530 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-15 23:51:45.084688956 +0000 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-15 23:47:20.342975822 +0000 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-15 23:47:20.342975822 +0000 2024-09-15T23:52:07.585 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-15 23:47:20.342975822 +0000 2024-09-15T23:52:07.586 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T23:52:07.649 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-15T23:52:07.649 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-15T23:52:07.649 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000205529 s, 2.5 MB/s 2024-09-15T23:52:07.650 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T23:52:07.707 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-09-15T23:52:07.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T23:52:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:08.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:08 smithi081 ceph-mon[35060]: Deploying daemon mgr.z on smithi102 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:08 smithi031 ceph-mon[31203]: Deploying daemon mgr.z on smithi102 2024-09-15T23:52:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:08 smithi102 ceph-mon[34458]: Deploying daemon mgr.z on smithi102 2024-09-15T23:52:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:09 smithi081 ceph-mon[35060]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:09.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:09 smithi031 ceph-mon[31203]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:09.344 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-mon[34458]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:09.344 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:09.340+0000 7f77118af040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:52:09.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:09.487+0000 7f77118af040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:52:09.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:09.552+0000 7f77118af040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:52:10.120 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:09.698+0000 7f77118af040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:52:10.120 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:09.836+0000 7f77118af040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:52:10.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:10.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:10.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:10 smithi031 ceph-mon[31203]: Deploying daemon mgr.y on smithi081 2024-09-15T23:52:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:52:10.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-09-15T23:52:10.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:10.148+0000 7f77118af040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:52:10.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:10.209+0000 7f77118af040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:52:10.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:10.397+0000 7f77118af040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:10.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:10.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:10 smithi102 ceph-mon[34458]: Deploying daemon mgr.y on smithi081 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:10.648 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:10.649 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:10 smithi081 ceph-mon[35060]: Deploying daemon mgr.y on smithi081 2024-09-15T23:52:10.695 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:52:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 -- 172.21.15.31:0/3409729468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 msgr2=0x7f32941073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 --2- 172.21.15.31:0/3409729468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941073c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3284009900 tx=0x7f328402a410 comp rx=0 tx=0).stop 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 -- 172.21.15.31:0/3409729468 shutdown_connections 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 --2- 172.21.15.31:0/3409729468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3294107900 0x7f3294109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 --2- 172.21.15.31:0/3409729468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941073c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 --2- 172.21.15.31:0/3409729468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32941026a0 0x7f3294104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.130+0000 7f329c707640 1 -- 172.21.15.31:0/3409729468 >> 172.21.15.31:0/3409729468 conn(0x7f32940fc2a0 msgr2=0x7f32940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.131+0000 7f329c707640 1 -- 172.21.15.31:0/3409729468 shutdown_connections 2024-09-15T23:52:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.131+0000 7f329c707640 1 -- 172.21.15.31:0/3409729468 wait complete. 2024-09-15T23:52:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.131+0000 7f329c707640 1 Processor -- start 2024-09-15T23:52:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.131+0000 7f329c707640 1 -- start start 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 0x7f3294197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3294107900 0x7f329419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f329410d780 con 0x7f3294104fd0 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f329410d600 con 0x7f3294107900 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329c707640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f329410d900 con 0x7f32941026a0 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f3299ca2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 0x7f3294197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f3299ca2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35870/0 (socket says 172.21.15.31:35870) 2024-09-15T23:52:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f3299ca2640 1 -- 172.21.15.31:0/4293299285 learned_addr learned my addr 172.21.15.31:0/4293299285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329aca4640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3294107900 0x7f329419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 -- 172.21.15.31:0/4293299285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3294107900 msgr2=0x7f329419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3294107900 0x7f329419f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 -- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 msgr2=0x7f32941983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 -- 172.21.15.31:0/4293299285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329419fac0 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f3299ca2640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f329a4a3640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 0x7f3294197e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f329000d6d0 tx=0x7f329000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.132+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290017070 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.133+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329419fdb0 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.133+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3290009820 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.133+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290004c40 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.133+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a0260 con 0x7f32941026a0 2024-09-15T23:52:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.133+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3290010670 con 0x7f32941026a0 2024-09-15T23:52:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.134+0000 7f328b7fe640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 0x7f328003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.134+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32940fdd90 con 0x7f32941026a0 2024-09-15T23:52:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.134+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f329004c310 con 0x7f32941026a0 2024-09-15T23:52:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.134+0000 7f3299ca2640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 0x7f328003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.134+0000 7f3299ca2640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 0x7f328003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3284000c00 tx=0x7f3284005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.139+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3290016c20 con 0x7f32941026a0 2024-09-15T23:52:11.307 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 systemd[1]: Starting Ceph mgr.y for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:52:11.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:11.325+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3294062630 con 0x7f3280039500 2024-09-15T23:52:11.594 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 podman[36408]: 2024-09-15 23:52:11.223284846 +0000 UTC m=+0.015548640 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:52:11.594 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 podman[36408]: 2024-09-15 23:52:11.332800277 +0000 UTC m=+0.125064067 container create 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:52:11.594 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 podman[36408]: 2024-09-15 23:52:11.526936193 +0000 UTC m=+0.319199984 container init 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-15T23:52:11.594 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 podman[36408]: 2024-09-15 23:52:11.529643482 +0000 UTC m=+0.321907282 container start 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T23:52:11.594 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 ceph-mgr[36427]: -- 172.21.15.81:0/2207486895 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b1fefd0e0 con 0x558b1feda800 2024-09-15T23:52:11.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:11.278+0000 7f77118af040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:52:11.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:11.346+0000 7f77118af040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:52:11.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 bash[36408]: 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f 2024-09-15T23:52:11.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 systemd[1]: Started Ceph mgr.y for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:52:11.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:11.666+0000 7fab70fc9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T23:52:11.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:11.728+0000 7fab70fc9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T23:52:11.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:11.884+0000 7fab70fc9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T23:52:12.261 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:11.953+0000 7fab70fc9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T23:52:12.261 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:12.112+0000 7fab70fc9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:52:12.262 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:12.260+0000 7fab70fc9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.091+0000 7f77118af040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.226+0000 7f77118af040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:12.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.367 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:12.528 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:12.528 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:12.528 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:12.528 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:12.528 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:12.529 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.529 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.529 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.529 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:12.529 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:12.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.372+0000 7f77118af040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:52:12.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.437+0000 7f77118af040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:52:12.621 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.596+0000 7f77118af040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:52:12.807 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:12.586+0000 7fab70fc9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T23:52:12.807 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:12.648+0000 7fab70fc9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T23:52:12.958 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.666+0000 7f77118af040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:52:12.958 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.726+0000 7f77118af040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:52:12.959 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.892+0000 7f77118af040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:52:12.959 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:12.957+0000 7f77118af040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:52:13.228 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:13.027+0000 7f77118af040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:52:13.229 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:13.110+0000 7f77118af040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:52:13.229 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:13.170+0000 7f77118af040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:52:13.229 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:13.227+0000 7f77118af040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:52:13.306 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:12.836+0000 7fab70fc9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:13.727+0000 7fab70fc9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:13.794+0000 7fab70fc9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: Reconfiguring daemon mgr.x on smithi031 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]: dispatch 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]': finished 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T23:52:14.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-15T23:52:14.012+0000 7f77118af040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: Reconfiguring daemon mgr.x on smithi031 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]: dispatch 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]': finished 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T23:52:14.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: Reconfiguring daemon mgr.x on smithi031 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]: dispatch 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3458733975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b544a11-7388-485f-9e4b-49e5bfa8459b"}]': finished 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T23:52:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:14.767 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:14.512+0000 7fab70fc9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T23:52:14.767 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:14.626+0000 7fab70fc9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T23:52:14.767 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:14.765+0000 7fab70fc9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T23:52:14.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:14.893+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f3290023ca0 con 0x7f32941026a0 2024-09-15T23:52:15.047 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:15.047 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: Standby manager daemon z started 2024-09-15T23:52:15.047 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1829983282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:14.825+0000 7fab70fc9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:14.976+0000 7fab70fc9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T23:52:15.048 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.045+0000 7fab70fc9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: Standby manager daemon z started 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T23:52:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:15.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1829983282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:15.307 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.105+0000 7fab70fc9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T23:52:15.307 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.263+0000 7fab70fc9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T23:52:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: Standby manager daemon z started 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/1761113869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1829983282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:15.597 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.329+0000 7fab70fc9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T23:52:15.597 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.396+0000 7fab70fc9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T23:52:15.597 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.478+0000 7fab70fc9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T23:52:15.597 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.537+0000 7fab70fc9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T23:52:15.597 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:15.595+0000 7fab70fc9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T23:52:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:15 smithi031 ceph-mon[31203]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T23:52:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:15 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T23:52:16.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-mon[35060]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T23:52:16.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:15 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T23:52:16.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:15 smithi102 ceph-mon[34458]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T23:52:16.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:15 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T23:52:16.807 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-15T23:52:16.376+0000 7fab70fc9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T23:52:17.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:17.155+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mon.1 v2:172.21.15.102:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f329000f3c0 con 0x7f32941026a0 2024-09-15T23:52:17.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: Standby manager daemon y started 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T23:52:17.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:16 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: Standby manager daemon y started 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T23:52:17.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:16 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: Standby manager daemon y started 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T23:52:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:16 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/2633094008' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T23:52:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:18 smithi031 ceph-mon[31203]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:18 smithi031 ceph-mon[31203]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T23:52:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T23:52:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:18 smithi081 ceph-mon[35060]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:18 smithi081 ceph-mon[35060]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T23:52:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T23:52:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:18 smithi102 ceph-mon[34458]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:18 smithi102 ceph-mon[34458]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T23:52:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T23:52:20.290 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:20 smithi031 ceph-mon[31203]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:20.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:20 smithi081 ceph-mon[35060]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:20.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:20 smithi102 ceph-mon[34458]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:21.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:21 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:21 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:21 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:21.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:21 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:21 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:21 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:21.777 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:21 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.777 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:21 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:21.777 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:21 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:22.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:22 smithi031 ceph-mon[31203]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:22.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:22.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:22.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:22.726 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:22 smithi081 ceph-mon[35060]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:22.726 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:22.726 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:22.726 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:22.763 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:22 smithi102 ceph-mon[34458]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:22.764 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:22.764 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:22.764 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T23:52:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:23 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:23 smithi031 ceph-mon[31203]: Deploying daemon osd.0 on smithi031 2024-09-15T23:52:23.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:23 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T23:52:23.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:23 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:23.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:23 smithi081 ceph-mon[35060]: Deploying daemon osd.0 on smithi031 2024-09-15T23:52:23.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:23 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T23:52:23.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:23 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:23.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:23 smithi102 ceph-mon[34458]: Deploying daemon osd.0 on smithi031 2024-09-15T23:52:24.439 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:24 smithi031 ceph-mon[31203]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:24.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:24 smithi081 ceph-mon[35060]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:24.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:24 smithi102 ceph-mon[34458]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:26.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:26 smithi031 ceph-mon[31203]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:26.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:26 smithi081 ceph-mon[35060]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:26.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:26 smithi102 ceph-mon[34458]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:28.490 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:28 smithi031 ceph-mon[31203]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:28.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:28 smithi081 ceph-mon[35060]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:28.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:28 smithi102 ceph-mon[34458]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:29.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:29.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:29.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:29.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:29 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:29.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:29 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:29.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:29 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:29.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:29 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:29.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:29 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:29.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:29 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:30.892 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:30 smithi031 ceph-mon[31203]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:31.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:30 smithi102 ceph-mon[34458]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:31.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:30 smithi081 ceph-mon[35060]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:32.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:32.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:32.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:32.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:32.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.044+0000 7f328b7fe640 1 -- 172.21.15.31:0/4293299285 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3294062630 con 0x7f3280039500 2024-09-15T23:52:33.045 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-09-15T23:52:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 msgr2=0x7f328003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 0x7f328003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3284000c00 tx=0x7f3284005a20 comp rx=0 tx=0).stop 2024-09-15T23:52:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 msgr2=0x7f3294197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 0x7f3294197e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f329000d6d0 tx=0x7f329000db90 comp rx=0 tx=0).stop 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 shutdown_connections 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3280039500 0x7f328003b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3294107900 0x7f329419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294104fd0 0x7f32941983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 --2- 172.21.15.31:0/4293299285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32941026a0 0x7f3294197e90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.048+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 >> 172.21.15.31:0/4293299285 conn(0x7f32940fc2a0 msgr2=0x7f3294103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.049+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 shutdown_connections 2024-09-15T23:52:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:52:33.049+0000 7f329c707640 1 -- 172.21.15.31:0/4293299285 wait complete. 2024-09-15T23:52:33.349 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:33.349 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.349 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:33.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:33.649 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.0.service 2024-09-15T23:52:33.651 INFO:tasks.cephadm:Deploying osd.1 on smithi081 with /dev/vg_nvme/lv_4... 2024-09-15T23:52:33.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T23:52:33.892 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:52:35.113 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:35 smithi031 ceph-mon[31203]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:35.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:35 smithi031 ceph-mon[31203]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T23:52:35.114 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:52:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-15T23:52:34.914+0000 7f009c452640 -1 osd.0 0 log_to_monitors true 2024-09-15T23:52:35.224 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:35 smithi081 ceph-mon[35060]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:35.224 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:35 smithi081 ceph-mon[35060]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T23:52:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:35 smithi102 ceph-mon[34458]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:35 smithi102 ceph-mon[34458]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T23:52:36.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-15T23:52:36.050+0000 7f00983d5640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T23:52:36.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-15T23:52:36.054+0000 7f00941fd640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:52:36.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T23:52:36.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.260 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:52:36.282 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T23:52:36.283 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: from='osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T23:52:36.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:36.546 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:52:36.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- 172.21.15.81:0/1221882527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 msgr2=0x7fbb94100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/1221882527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94100ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a410 comp rx=0 tx=0).stop 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- 172.21.15.81:0/1221882527 shutdown_connections 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/1221882527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb94108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/1221882527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 0x7fbb94102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/1221882527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94100ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- 172.21.15.81:0/1221882527 >> 172.21.15.81:0/1221882527 conn(0x7fbb940fc280 msgr2=0x7fbb940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- 172.21.15.81:0/1221882527 shutdown_connections 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- 172.21.15.81:0/1221882527 wait complete. 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 Processor -- start 2024-09-15T23:52:36.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.989+0000 7fbb9adb6640 1 -- start start 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 0x7fbb941030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb941035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410c630 con 0x7fbb94102690 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410c4b0 con 0x7fbb94100ad0 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb9adb6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9410c7b0 con 0x7fbb94101cd0 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60846/0 (socket says 172.21.15.81:60846) 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 -- 172.21.15.81:0/4122829041 learned_addr learned my addr 172.21.15.81:0/4122829041 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb93fff640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 0x7fbb941030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb99353640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb941035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 -- 172.21.15.81:0/4122829041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 msgr2=0x7fbb941030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:36.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 0x7fbb941030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:36.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 -- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 msgr2=0x7fbb941035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:36.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb941035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:36.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 -- 172.21.15.81:0/4122829041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb94103eb0 con 0x7fbb94100ad0 2024-09-15T23:52:36.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb99353640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb941035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:52:36.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.990+0000 7fbb98b52640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb880314e0 tx=0x7fbb88031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:36.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.991+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8803f030 con 0x7fbb94100ad0 2024-09-15T23:52:36.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.991+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb880044b0 con 0x7fbb94100ad0 2024-09-15T23:52:36.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.991+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a1fa0 con 0x7fbb94100ad0 2024-09-15T23:52:36.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.991+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88035070 con 0x7fbb94100ad0 2024-09-15T23:52:36.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.991+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a2340 con 0x7fbb94100ad0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.992+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb941a2630 con 0x7fbb94100ad0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.993+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fbb88042030 con 0x7fbb94100ad0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.994+0000 7fbb91ffb640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 0x7fbb840a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.994+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fbb880ee700 con 0x7fbb94100ad0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.994+0000 7fbb93fff640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 0x7fbb840a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.994+0000 7fbb93fff640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 0x7fbb840a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb941047e0 tx=0x7fbb8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:52:36.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:36.997+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb880bca80 con 0x7fbb94100ad0 2024-09-15T23:52:37.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:37.178+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbb941a2800 con 0x7fbb840a31b0 2024-09-15T23:52:37.179 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.180 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:52:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:38.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi031 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] boot 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:38.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi031 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] boot 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:38.286 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:38.287 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:38.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi031 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: osd.0 [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] boot 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:52:38.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:39.195 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:39 smithi081 ceph-mon[35060]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:39.195 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:39 smithi081 ceph-mon[35060]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:39.195 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:39 smithi081 ceph-mon[35060]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T23:52:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:39 smithi102 ceph-mon[34458]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:39 smithi102 ceph-mon[34458]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:39.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:39 smithi102 ceph-mon[34458]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T23:52:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:39 smithi031 ceph-mon[31203]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T23:52:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:39 smithi031 ceph-mon[31203]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:52:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:39 smithi031 ceph-mon[31203]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/3799035713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]': finished 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:40 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/2682640403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:40.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/3799035713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]': finished 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:52:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:40 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/2682640403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:40.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:40.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/3799035713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]: dispatch 2024-09-15T23:52:40.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc0ccb76-ecd7-4c5b-b4f4-f282127b892a"}]': finished 2024-09-15T23:52:40.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T23:52:40.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:52:40.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:40 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/2682640403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:52:42.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:42 smithi031 ceph-mon[31203]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:42.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:42 smithi081 ceph-mon[35060]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:42.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:42 smithi102 ceph-mon[34458]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:44.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:44 smithi081 ceph-mon[35060]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:44.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:44 smithi031 ceph-mon[31203]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:44.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:44 smithi102 ceph-mon[34458]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:46.378 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:46 smithi081 ceph-mon[35060]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:46.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:46 smithi031 ceph-mon[31203]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:46.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:46 smithi102 ceph-mon[34458]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:48.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:48 smithi031 ceph-mon[31203]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:48.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T23:52:48.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:48.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:48 smithi031 ceph-mon[31203]: Deploying daemon osd.1 on smithi081 2024-09-15T23:52:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:48 smithi081 ceph-mon[35060]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T23:52:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:48 smithi081 ceph-mon[35060]: Deploying daemon osd.1 on smithi081 2024-09-15T23:52:48.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:48 smithi102 ceph-mon[34458]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:48.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T23:52:48.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:48.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:48 smithi102 ceph-mon[34458]: Deploying daemon osd.1 on smithi081 2024-09-15T23:52:50.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:50 smithi031 ceph-mon[31203]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:50.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:50 smithi081 ceph-mon[35060]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:50.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:50 smithi102 ceph-mon[34458]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:52.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:52 smithi031 ceph-mon[31203]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:52.513 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:52 smithi081 ceph-mon[35060]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:52.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:52 smithi102 ceph-mon[34458]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:54.469 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:54 smithi081 ceph-mon[35060]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:54.469 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:54 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:54.469 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:54 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:54.469 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:54 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:54.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:54 smithi031 ceph-mon[31203]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:54.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:54 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:54.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:54 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:54.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:54 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:54.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:54 smithi102 ceph-mon[34458]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:54.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:54 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:54.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:54 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:54.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:54 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:56.248 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:56.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:56.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:56.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:56.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:56.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:52:56.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:52:56.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:57.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.948+0000 7fbb91ffb640 1 -- 172.21.15.81:0/4122829041 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb941a2800 con 0x7fbb840a31b0 2024-09-15T23:52:57.949 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 1 on host 'smithi081' 2024-09-15T23:52:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 msgr2=0x7fbb840a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 0x7fbb840a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb941047e0 tx=0x7fbb8000b390 comp rx=0 tx=0).stop 2024-09-15T23:52:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 msgr2=0x7fbb94104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:52:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb880314e0 tx=0x7fbb88031a30 comp rx=0 tx=0).stop 2024-09-15T23:52:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 shutdown_connections 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbb840a31b0 0x7fbb840a5670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb94102690 0x7fbb941035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb94101cd0 0x7fbb941030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 --2- 172.21.15.81:0/4122829041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb94100ad0 0x7fbb94104a60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 >> 172.21.15.81:0/4122829041 conn(0x7fbb940fc280 msgr2=0x7fbb94069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 shutdown_connections 2024-09-15T23:52:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:52:57.950+0000 7fbb9adb6640 1 -- 172.21.15.81:0/4122829041 wait complete. 2024-09-15T23:52:58.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:58 smithi031 ceph-mon[31203]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:58.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:58.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:52:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.522 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:58 smithi081 ceph-mon[35060]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:58.523 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:58.523 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.523 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:52:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.550 DEBUG:teuthology.orchestra.run.smithi081:osd.1> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.1.service 2024-09-15T23:52:58.553 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_4... 2024-09-15T23:52:58.553 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T23:52:58.580 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:58 smithi102 ceph-mon[34458]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:52:58.580 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:52:58.580 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.581 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:52:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:52:58.800 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:53:00.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:00 smithi081 ceph-mon[35060]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:00.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:00 smithi081 ceph-mon[35060]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:00.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:00 smithi081 ceph-mon[35060]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:00.307 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-15T23:53:00.017+0000 7fcf72ad1640 -1 osd.1 0 log_to_monitors true 2024-09-15T23:53:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:00 smithi102 ceph-mon[34458]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:00 smithi102 ceph-mon[34458]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:00.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:00 smithi102 ceph-mon[34458]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:00.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:00 smithi031 ceph-mon[31203]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:00.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:00 smithi031 ceph-mon[31203]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:00.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:00 smithi031 ceph-mon[31203]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T23:53:01.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:53:01.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-09-15T23:53:01.344 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-mon[35060]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T23:53:01.344 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-mon[35060]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.344 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-mon[35060]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T23:53:01.344 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:01.345 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-mon[35060]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.455 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:53:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:01 smithi031 ceph-mon[31203]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T23:53:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:01 smithi031 ceph-mon[31203]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:01 smithi031 ceph-mon[31203]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T23:53:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:01 smithi031 ceph-mon[31203]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.574 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:01 smithi102 ceph-mon[34458]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T23:53:01.575 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:01 smithi102 ceph-mon[34458]: from='osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.575 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:01 smithi102 ceph-mon[34458]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T23:53:01.575 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:01.575 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:01 smithi102 ceph-mon[34458]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 -- 172.21.15.102:0/2076274403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 msgr2=0x7f57d810bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 --2- 172.21.15.102:0/2076274403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d810bf30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ca0 tx=0x7f57d002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 -- 172.21.15.102:0/2076274403 shutdown_connections 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 --2- 172.21.15.102:0/2076274403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d810bf30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 --2- 172.21.15.102:0/2076274403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 --2- 172.21.15.102:0/2076274403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 0x7f57d8074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 -- 172.21.15.102:0/2076274403 >> 172.21.15.102:0/2076274403 conn(0x7f57d80fd390 msgr2=0x7f57d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 -- 172.21.15.102:0/2076274403 shutdown_connections 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.926+0000 7f57de88d640 1 -- 172.21.15.102:0/2076274403 wait complete. 2024-09-15T23:53:01.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 Processor -- start 2024-09-15T23:53:01.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 -- start start 2024-09-15T23:53:01.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:01.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 0x7f57d819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:01.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810f900 con 0x7f57d8074b40 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810f780 con 0x7f57d80761a0 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.927+0000 7f57de88d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810fa80 con 0x7f57d8109a70 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34136/0 (socket says 172.21.15.102:34136) 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 -- 172.21.15.102:0/4156392985 learned_addr learned my addr 172.21.15.102:0/4156392985 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d77fe640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 0x7f57d819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d7fff640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 -- 172.21.15.102:0/4156392985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 msgr2=0x7f57d819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 0x7f57d819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 -- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 msgr2=0x7f57d8199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:01.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 -- 172.21.15.102:0/4156392985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81a1b30 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d7fff640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57dce2a640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009dc0 tx=0x7f57d0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004040 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57d0004690 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81a1dc0 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d002f4c0 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.928+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a2270 con 0x7f57d8109a70 2024-09-15T23:53:01.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.929+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8108880 con 0x7f57d8109a70 2024-09-15T23:53:01.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.930+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f57d0030650 con 0x7f57d8109a70 2024-09-15T23:53:01.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.931+0000 7f57d57fa640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 0x7f57c00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:01.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.931+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f57d00ef6a0 con 0x7f57d8109a70 2024-09-15T23:53:01.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.931+0000 7f57d7fff640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 0x7f57c00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:01.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.932+0000 7f57d7fff640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 0x7f57c00a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57c800abe0 tx=0x7f57c800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:01.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:01.935+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57d00bdd30 con 0x7f57d8109a70 2024-09-15T23:53:02.057 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-15T23:53:01.607+0000 7fcf6ea54640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T23:53:02.057 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-15T23:53:01.610+0000 7fcf66874640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:53:02.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:02.110+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f57d8062630 con 0x7f57c00a31e0 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi081 2024-09-15T23:53:02.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='osd.1 ' entity='osd.1' 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:02.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:02.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:02.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:53:02.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi081 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='osd.1 ' entity='osd.1' 2024-09-15T23:53:02.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:02.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:02.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi081 2024-09-15T23:53:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 17088M 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='osd.1 ' entity='osd.1' 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:02.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:02.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] boot 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T23:53:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:03 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:03.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:53:03.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:53:03.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:03.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:03.469 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] boot 2024-09-15T23:53:03.469 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T23:53:03.469 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:03 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:53:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:53:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: osd.1 [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] boot 2024-09-15T23:53:03.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T23:53:03.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:03 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T23:53:04.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/2576631674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]': finished 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T23:53:04.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T23:53:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/2576631674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:05.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]': finished 2024-09-15T23:53:05.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T23:53:05.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/2576631674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]: dispatch 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16d860f1-d8da-4d5d-ae50-10e77a4c4b2c"}]': finished 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T23:53:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:06.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:05 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1735815481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:06.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:05 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/1735815481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:06.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:05 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1735815481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:07.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:06 smithi031 ceph-mon[31203]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:07.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:06 smithi081 ceph-mon[35060]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:07.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:06 smithi102 ceph-mon[34458]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:09.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:08 smithi031 ceph-mon[31203]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:09.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:08 smithi081 ceph-mon[35060]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:09.067 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:08 smithi102 ceph-mon[34458]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:10.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:10 smithi102 ceph-mon[34458]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:11.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:10 smithi031 ceph-mon[31203]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:10 smithi081 ceph-mon[35060]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:12.959 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:12 smithi102 ceph-mon[34458]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:12.959 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T23:53:12.959 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:12 smithi031 ceph-mon[31203]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T23:53:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:12 smithi081 ceph-mon[35060]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T23:53:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:13.847 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:13 smithi102 ceph-mon[34458]: Deploying daemon osd.2 on smithi102 2024-09-15T23:53:14.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:13 smithi031 ceph-mon[31203]: Deploying daemon osd.2 on smithi102 2024-09-15T23:53:14.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:13 smithi081 ceph-mon[35060]: Deploying daemon osd.2 on smithi102 2024-09-15T23:53:14.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:14 smithi102 ceph-mon[34458]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:15.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:14 smithi031 ceph-mon[31203]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:15.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:14 smithi081 ceph-mon[35060]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:16.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:16 smithi102 ceph-mon[34458]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:17.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:16 smithi031 ceph-mon[31203]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:17.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:16 smithi081 ceph-mon[35060]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:18.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:18 smithi102 ceph-mon[34458]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:18.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:18.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:18.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:19.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:18 smithi031 ceph-mon[31203]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:19.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:19.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:19.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:19.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:18 smithi081 ceph-mon[35060]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:19.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:19.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:19.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:21.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:21.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:21.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:21.229 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:21.230 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:22.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.489+0000 7f57d57fa640 1 -- 172.21.15.102:0/4156392985 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57d8062630 con 0x7f57c00a31e0 2024-09-15T23:53:22.491 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-09-15T23:53:22.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.491+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 msgr2=0x7f57c00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:22.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.491+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 0x7f57c00a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57c800abe0 tx=0x7f57c800a5c0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.491+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 msgr2=0x7f57d81a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:22.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.491+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009dc0 tx=0x7f57d0003ba0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 shutdown_connections 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f57c00a31e0 0x7f57c00a56a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d8109a70 0x7f57d81a14c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57d80761a0 0x7f57d819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 --2- 172.21.15.102:0/4156392985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57d8074b40 0x7f57d8199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 >> 172.21.15.102:0/4156392985 conn(0x7f57d80fd390 msgr2=0x7f57d8105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 shutdown_connections 2024-09-15T23:53:22.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:53:22.492+0000 7f57de88d640 1 -- 172.21.15.102:0/4156392985 wait complete. 2024-09-15T23:53:22.794 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:22 smithi102 ceph-mon[34458]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:22.795 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:22.795 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:22.795 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:22 smithi031 ceph-mon[31203]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:22 smithi081 ceph-mon[35060]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:23.080 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.2.service 2024-09-15T23:53:23.082 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-09-15T23:53:23.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T23:53:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:53:24.595 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-15T23:53:24.479+0000 7fc1204b1640 -1 osd.2 0 log_to_monitors true 2024-09-15T23:53:24.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:24 smithi031 ceph-mon[31203]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:24.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:24 smithi031 ceph-mon[31203]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:24.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:24 smithi031 ceph-mon[31203]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:24.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:24 smithi102 ceph-mon[34458]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:24.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:24 smithi102 ceph-mon[34458]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:24.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:24 smithi102 ceph-mon[34458]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:25.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:24 smithi081 ceph-mon[35060]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:25.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:24 smithi081 ceph-mon[35060]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:25.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:24 smithi081 ceph-mon[35060]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T23:53:25.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:53:25.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-09-15T23:53:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:25 smithi031 ceph-mon[31203]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T23:53:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:25 smithi031 ceph-mon[31203]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T23:53:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:25 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:25 smithi031 ceph-mon[31203]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:25.925 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:25 smithi031 ceph-mon[31203]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:26.043 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-15T23:53:25.672+0000 7fc11c434640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T23:53:26.043 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-15T23:53:25.675+0000 7fc11825c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:53:26.043 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-mon[34458]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T23:53:26.043 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-mon[34458]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T23:53:26.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:26.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-mon[34458]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:26.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:25 smithi102 ceph-mon[34458]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:26.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:25 smithi081 ceph-mon[35060]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T23:53:26.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:25 smithi081 ceph-mon[35060]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T23:53:26.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:25 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:26.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:25 smithi081 ceph-mon[35060]: from='osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:26.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:25 smithi081 ceph-mon[35060]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:53:26.162 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 -- 172.21.15.31:0/383272457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 msgr2=0x7f38c4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/383272457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c4100f10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009900 tx=0x7f38b802a410 comp rx=0 tx=0).stop 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 -- 172.21.15.31:0/383272457 shutdown_connections 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/383272457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c41026d0 0x7f38c4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/383272457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c4101d10 0x7f38c4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/383272457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c4100f10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 -- 172.21.15.31:0/383272457 >> 172.21.15.31:0/383272457 conn(0x7f38c40fc2a0 msgr2=0x7f38c40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:53:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 -- 172.21.15.31:0/383272457 shutdown_connections 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.634+0000 7f38cb0fb640 1 -- 172.21.15.31:0/383272457 wait complete. 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 Processor -- start 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 -- start start 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 0x7f38c4109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 0x7f38c410a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d770 con 0x7f38c4100b10 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d5f0 con 0x7f38c4101d10 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38cb0fb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d8f0 con 0x7f38c41026d0 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c8e97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c8e97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47072/0 (socket says 172.21.15.31:47072) 2024-09-15T23:53:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c8e97640 1 -- 172.21.15.31:0/1249707513 learned_addr learned my addr 172.21.15.31:0/1249707513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c3fff640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 0x7f38c4109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c9698640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 0x7f38c410a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.635+0000 7f38c8e97640 1 -- 172.21.15.31:0/1249707513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 msgr2=0x7f38c410a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c8e97640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 0x7f38c410a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c8e97640 1 -- 172.21.15.31:0/1249707513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 msgr2=0x7f38c4109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c8e97640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 0x7f38c4109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c8e97640 1 -- 172.21.15.31:0/1249707513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c410a8e0 con 0x7f38c4100b10 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c9698640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 0x7f38c410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c3fff640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 0x7f38c4109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c8e97640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38b80314e0 tx=0x7f38b80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b803f030 con 0x7f38c4100b10 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b8003cb0 con 0x7f38c4100b10 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41a1f50 con 0x7f38c4100b10 2024-09-15T23:53:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8035070 con 0x7f38c4100b10 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.636+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a2340 con 0x7f38c4100b10 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.637+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4107f00 con 0x7f38c4100b10 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.638+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f38b8042030 con 0x7f38c4100b10 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.639+0000 7f38c1ffb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 0x7f38b40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.639+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f38b80ee870 con 0x7f38c4100b10 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.639+0000 7f38c3fff640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 0x7f38b40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.639+0000 7f38c3fff640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 0x7f38b40a56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38b00058c0 tx=0x7f38b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.644+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38b80bcc40 con 0x7f38c4100b10 2024-09-15T23:53:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:26.830+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f38c4062630 con 0x7f38b40a3230 2024-09-15T23:53:26.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:26.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:53:26.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T23:53:26.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:26.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi102 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:26.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi102 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:27.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi102 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:27.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:28.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:53:28.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] boot 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:28.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] boot 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:28.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:28.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: osd.2 [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] boot 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:28.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:28.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:28 smithi031 ceph-mon[31203]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:28.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:28 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:28.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:28 smithi031 ceph-mon[31203]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T23:53:28.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:28 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:29.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:28 smithi081 ceph-mon[35060]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:29.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:28 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:29.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:28 smithi081 ceph-mon[35060]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T23:53:29.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:28 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:29.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:28 smithi102 ceph-mon[34458]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T23:53:29.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:28 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:29.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:28 smithi102 ceph-mon[34458]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T23:53:29.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:28 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:53:29.504 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:29.504 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:29.504 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:29.505 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49628]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:29.698+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f38b80f0ce0 con 0x7f38c4100b10 2024-09-15T23:53:29.790 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:29.790 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T23:53:29.790 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]: dispatch 2024-09-15T23:53:29.790 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]': finished 2024-09-15T23:53:29.791 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T23:53:29.791 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:29 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:29.791 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49662]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:29.791 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49662]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:29.791 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49662]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:29.791 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 15 23:53:29 smithi031 sudo[49662]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]: dispatch 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]': finished 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T23:53:29.955 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:29 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:30.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:53:30.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T23:53:30.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]: dispatch 2024-09-15T23:53:30.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1910340745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747"}]': finished 2024-09-15T23:53:30.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T23:53:30.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:29 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:30.210 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:29 smithi081 sudo[43203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:30.211 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:29 smithi081 sudo[43203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:30.211 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:29 smithi081 sudo[43203]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:30.211 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:29 smithi081 sudo[43203]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:30.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:30 smithi081 sudo[43206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:30.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:30 smithi081 sudo[43206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:30.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:30 smithi081 sudo[43206]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:30.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 15 23:53:30 smithi081 sudo[43206]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:30.735 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43449]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:30.736 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43449]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:30.736 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43449]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:30.736 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43449]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:30.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:30 smithi102 ceph-mon[34458]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:30.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:30 smithi102 ceph-mon[34458]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T23:53:30.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:30.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:30 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2626099348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:30.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:30 smithi031 ceph-mon[31203]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:30 smithi031 ceph-mon[31203]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T23:53:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:30 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2626099348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:30 smithi081 ceph-mon[35060]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:30 smithi081 ceph-mon[35060]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T23:53:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:30 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2626099348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:31.120 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43452]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:31.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43452]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:31.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43452]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:31.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 15 23:53:30 smithi102 sudo[43452]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50080]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50080]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:32.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:32.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:32.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50084]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:32.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:31 smithi031 sudo[50084]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:32.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:31 smithi102 sudo[43455]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:32.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:31 smithi102 sudo[43455]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:32.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:31 smithi102 sudo[43455]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:32.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:31 smithi102 sudo[43455]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:32.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 sudo[43458]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:32.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 sudo[43458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 sudo[43458]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:32.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 sudo[43458]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T23:53:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:32.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43209]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:32.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43209]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:33.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:33.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:33.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43212]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 sudo[43212]: pam_unix(sudo:session): session closed for user root 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:33.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:33.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:34.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:33 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:34.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:33 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T23:53:34.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:33 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T23:53:34.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:34 smithi102 ceph-mon[34458]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:35.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:34 smithi031 ceph-mon[31203]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:35.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:34 smithi081 ceph-mon[35060]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:36.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:36 smithi031 ceph-mon[31203]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:37.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:36 smithi081 ceph-mon[35060]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:37.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:36 smithi102 ceph-mon[34458]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:38 smithi031 ceph-mon[31203]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T23:53:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:38 smithi031 ceph-mon[31203]: Deploying daemon osd.3 on smithi031 2024-09-15T23:53:39.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:38 smithi081 ceph-mon[35060]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:39.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T23:53:39.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:39.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:38 smithi081 ceph-mon[35060]: Deploying daemon osd.3 on smithi031 2024-09-15T23:53:39.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:38 smithi102 ceph-mon[34458]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:39.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T23:53:39.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:39.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:38 smithi102 ceph-mon[34458]: Deploying daemon osd.3 on smithi031 2024-09-15T23:53:40.712 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:40 smithi031 ceph-mon[31203]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:41.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:40 smithi081 ceph-mon[35060]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:41.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:40 smithi102 ceph-mon[34458]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:42.797 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:42 smithi031 ceph-mon[31203]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:43.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:42 smithi081 ceph-mon[35060]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:43.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:42 smithi102 ceph-mon[34458]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:44.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:44 smithi031 ceph-mon[31203]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:44.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:44.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:44.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:44 smithi081 ceph-mon[35060]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:44 smithi102 ceph-mon[34458]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:47.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:46 smithi031 ceph-mon[31203]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:47.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:46 smithi081 ceph-mon[35060]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:47.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:46 smithi102 ceph-mon[34458]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:48.174 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.174 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.174 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:48.174 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:48.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.212+0000 7f38c1ffb640 1 -- 172.21.15.31:0/1249707513 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38c4062630 con 0x7f38b40a3230 2024-09-15T23:53:48.214 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.214+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 msgr2=0x7f38b40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.214+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 0x7f38b40a56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38b00058c0 tx=0x7f38b0005850 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.214+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 msgr2=0x7f38c410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.214+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38b80314e0 tx=0x7f38b80042e0 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 shutdown_connections 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f38b40a3230 0x7f38b40a56f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38c41026d0 0x7f38c410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38c4101d10 0x7f38c4109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 --2- 172.21.15.31:0/1249707513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4100b10 0x7f38c410b490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 >> 172.21.15.31:0/1249707513 conn(0x7f38c40fc2a0 msgr2=0x7f38c41050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 shutdown_connections 2024-09-15T23:53:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:53:48.215+0000 7f38cb0fb640 1 -- 172.21.15.31:0/1249707513 wait complete. 2024-09-15T23:53:48.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:48.866 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.3.service 2024-09-15T23:53:48.908 INFO:tasks.cephadm:Deploying osd.4 on smithi081 with /dev/vg_nvme/lv_3... 2024-09-15T23:53:48.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T23:53:49.161 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:53:49.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:48 smithi031 ceph-mon[31203]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:49.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:49.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:49.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:49.282 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:48 smithi081 ceph-mon[35060]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:49.282 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:49.282 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:49.282 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:48 smithi102 ceph-mon[34458]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:49.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:53:49.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:49.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:50.068 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 15 23:53:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-15T23:53:49.951+0000 7fda1141f640 -1 osd.3 0 log_to_monitors true 2024-09-15T23:53:51.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:50 smithi031 ceph-mon[31203]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:51.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:50 smithi031 ceph-mon[31203]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T23:53:51.234 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:50 smithi081 ceph-mon[35060]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:51.235 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:50 smithi081 ceph-mon[35060]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T23:53:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:50 smithi102 ceph-mon[34458]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:50 smithi102 ceph-mon[34458]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T23:53:51.978 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:53:52.004 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2024-09-15T23:53:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T23:53:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T23:53:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.255 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:52.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:52.273 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:53:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:53:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:53:52.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-15T23:53:52.297+0000 7fda0d3a2640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T23:53:52.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-15T23:53:52.303+0000 7fda051c2640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:53:52.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/3672027959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0102690 msgr2=0x7fc3d0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:52.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/3672027959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0102690 0x7fc3d0108f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009ca0 tx=0x7fc3c802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/3672027959 shutdown_connections 2024-09-15T23:53:52.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/3672027959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0102690 0x7fc3d0108f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/3672027959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0101cd0 0x7fc3d0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/3672027959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 0x7fc3d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/3672027959 >> 172.21.15.81:0/3672027959 conn(0x7fc3d00fc280 msgr2=0x7fc3d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/3672027959 shutdown_connections 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.735+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/3672027959 wait complete. 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 Processor -- start 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 -- start start 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 0x7fc3d0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0102690 0x7fc3d01a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c630 con 0x7fc3d0102690 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c4b0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d7b0e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c7b0 con 0x7fc3d0100ad0 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d50a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d50a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37606/0 (socket says 172.21.15.81:37606) 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d50a9640 1 -- 172.21.15.81:0/2908441382 learned_addr learned my addr 172.21.15.81:0/2908441382 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d58aa640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 0x7fc3d0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.736+0000 7fc3d60ab640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0102690 0x7fc3d01a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 -- 172.21.15.81:0/2908441382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 msgr2=0x7fc3d0199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 0x7fc3d0199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 -- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0102690 msgr2=0x7fc3d01a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0102690 0x7fc3d01a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 -- 172.21.15.81:0/2908441382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d01a1be0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d50a9640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000e9f0 tx=0x7fc3c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000cd80 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3c00108e0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d01a1ed0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0018920 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.737+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01a23d0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.738+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d01a2600 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.739+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc3c0010a90 con 0x7fc3d0101cd0 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.740+0000 7fc3c6ffd640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 0x7fc3bc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:53:52.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.740+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7fc3c00ceba0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.740+0000 7fc3d58aa640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 0x7fc3bc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:53:52.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.741+0000 7fc3d58aa640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 0x7fc3bc0a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005fd0 tx=0x7fc3cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:53:52.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.743+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3c009c5b0 con 0x7fc3d0101cd0 2024-09-15T23:53:52.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:53:52.924+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3d006a6b0 con 0x7fc3bc0a32f0 2024-09-15T23:53:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi031 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' 2024-09-15T23:53:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi031 2024-09-15T23:53:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-15T23:53:53.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:53:53.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T23:53:53.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' 2024-09-15T23:53:53.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:53.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi031 2024-09-15T23:53:53.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-15T23:53:53.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:53:53.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T23:53:53.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:53.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: from='osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469]' entity='osd.3' 2024-09-15T23:53:53.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:53:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:53:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] boot 2024-09-15T23:53:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:53 smithi031 ceph-mon[31203]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] boot 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:53 smithi081 ceph-mon[35060]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T23:53:54.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:53:54.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: osd.3 [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] boot 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:53:54.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:53 smithi102 ceph-mon[34458]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T23:53:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:54 smithi031 ceph-mon[31203]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:54 smithi031 ceph-mon[31203]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T23:53:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:54 smithi081 ceph-mon[35060]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:54 smithi081 ceph-mon[35060]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T23:53:55.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:54 smithi102 ceph-mon[34458]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-15T23:53:55.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:54 smithi102 ceph-mon[34458]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T23:53:56.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/3264573944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]': finished 2024-09-15T23:53:56.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T23:53:56.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:56.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:55 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/2966782584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/3264573944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]': finished 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:56.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:55 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/2966782584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:56.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/3264573944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]: dispatch 2024-09-15T23:53:56.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1085071-44fa-4277-b09b-7e5ef3b45330"}]': finished 2024-09-15T23:53:56.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T23:53:56.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:56.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:55 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/2966782584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:53:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:56 smithi031 ceph-mon[31203]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:53:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:56 smithi031 ceph-mon[31203]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T23:53:57.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:56 smithi081 ceph-mon[35060]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:53:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:56 smithi081 ceph-mon[35060]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T23:53:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:56 smithi102 ceph-mon[34458]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:53:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:56 smithi102 ceph-mon[34458]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T23:53:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:53:59.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:53:58 smithi031 ceph-mon[31203]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:53:59.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:53:58 smithi081 ceph-mon[35060]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:53:59.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:53:58 smithi102 ceph-mon[34458]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:01.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:00 smithi081 ceph-mon[35060]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-15T23:54:01.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:00 smithi031 ceph-mon[31203]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-15T23:54:01.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:00 smithi102 ceph-mon[34458]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-15T23:54:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:02 smithi031 ceph-mon[31203]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T23:54:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:02 smithi081 ceph-mon[35060]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T23:54:03.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:02 smithi102 ceph-mon[34458]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T23:54:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:04 smithi031 ceph-mon[31203]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:54:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T23:54:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:05.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:04 smithi031 ceph-mon[31203]: Deploying daemon osd.4 on smithi081 2024-09-15T23:54:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:04 smithi081 ceph-mon[35060]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:54:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T23:54:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:04 smithi081 ceph-mon[35060]: Deploying daemon osd.4 on smithi081 2024-09-15T23:54:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:04 smithi102 ceph-mon[34458]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:54:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T23:54:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:04 smithi102 ceph-mon[34458]: Deploying daemon osd.4 on smithi081 2024-09-15T23:54:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:06 smithi031 ceph-mon[31203]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T23:54:07.271 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:06 smithi081 ceph-mon[35060]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T23:54:07.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:06 smithi102 ceph-mon[34458]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T23:54:09.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:08 smithi031 ceph-mon[31203]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T23:54:09.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:08 smithi081 ceph-mon[35060]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T23:54:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:08 smithi102 ceph-mon[34458]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T23:54:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:10 smithi031 ceph-mon[31203]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:54:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:11.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:10 smithi081 ceph-mon[35060]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:54:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:10 smithi102 ceph-mon[34458]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:54:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:11.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:11.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:12 smithi031 ceph-mon[31203]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:13.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:12 smithi081 ceph-mon[35060]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:13.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:12 smithi102 ceph-mon[34458]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.309 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:14 smithi081 ceph-mon[35060]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:14.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:14.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:14.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:14 smithi102 ceph-mon[34458]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:14.393 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 4 on host 'smithi081' 2024-09-15T23:54:14.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.392+0000 7fc3c6ffd640 1 -- 172.21.15.81:0/2908441382 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3d006a6b0 con 0x7fc3bc0a32f0 2024-09-15T23:54:14.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 msgr2=0x7fc3bc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 0x7fc3bc0a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005fd0 tx=0x7fc3cc00b410 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 msgr2=0x7fc3d019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000e9f0 tx=0x7fc3c000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 shutdown_connections 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3bc0a32f0 0x7fc3bc0a57b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d0102690 0x7fc3d01a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3d0101cd0 0x7fc3d019a4f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 --2- 172.21.15.81:0/2908441382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3d0100ad0 0x7fc3d0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 >> 172.21.15.81:0/2908441382 conn(0x7fc3d00fc280 msgr2=0x7fc3d0069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:54:14.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 shutdown_connections 2024-09-15T23:54:14.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:54:14.395+0000 7fc3d7b0e640 1 -- 172.21.15.81:0/2908441382 wait complete. 2024-09-15T23:54:14.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:14.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:14.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:14.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:14 smithi031 ceph-mon[31203]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:15.133 DEBUG:teuthology.orchestra.run.smithi081:osd.4> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.4.service 2024-09-15T23:54:15.135 INFO:tasks.cephadm:Deploying osd.5 on smithi102 with /dev/vg_nvme/lv_3... 2024-09-15T23:54:15.135 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T23:54:15.383 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:54:15.430 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:15 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:15.430 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:15 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:15.430 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:15 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:15 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:15 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:15 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:15 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:15 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:15 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:16.391 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:16 smithi081 ceph-mon[35060]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:16.391 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 15 23:54:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-15T23:54:16.243+0000 7f8ea4fbc640 -1 osd.4 0 log_to_monitors true 2024-09-15T23:54:16.464 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:16 smithi102 ceph-mon[34458]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:16.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:16 smithi031 ceph-mon[31203]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:17 smithi102 ceph-mon[34458]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:17 smithi102 ceph-mon[34458]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:17 smithi031 ceph-mon[31203]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:17 smithi031 ceph-mon[31203]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:17 smithi081 ceph-mon[35060]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:17 smithi081 ceph-mon[35060]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T23:54:17.926 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:54:17.953 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-09-15T23:54:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:54:18.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-15T23:54:18.159+0000 7f8ea0f3f640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T23:54:18.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-15T23:54:18.167+0000 7f8e9cd67640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi081 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:18.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi081 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-15T23:54:18.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:18.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:18.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi081 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 8544M 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:18.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:18.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:18.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1367194985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 msgr2=0x7f6b28074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:18.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1367194985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b28074680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009900 tx=0x7f6b2402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1367194985 shutdown_connections 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1367194985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 0x7f6b281105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1367194985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 0x7f6b280750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1367194985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b28074680 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1367194985 >> 172.21.15.102:0/1367194985 conn(0x7f6b280fd390 msgr2=0x7f6b280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1367194985 shutdown_connections 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.658+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1367194985 wait complete. 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 Processor -- start 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 -- start start 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 0x7f6b2819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:18.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 0x7f6b281a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28113d50 con 0x7f6b28074c50 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28113bd0 con 0x7f6b2810e1e0 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2faf7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28113ed0 con 0x7f6b28076220 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45056/0 (socket says 172.21.15.102:45056) 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 -- 172.21.15.102:0/1788209807 learned_addr learned my addr 172.21.15.102:0/1788209807 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2e094640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 0x7f6b281a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 -- 172.21.15.102:0/1788209807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 msgr2=0x7f6b281a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 0x7f6b281a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d893640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 0x7f6b2819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 -- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 msgr2=0x7f6b2819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:18.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 0x7f6b2819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.659+0000 7f6b2d092640 1 -- 172.21.15.102:0/1788209807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b281a5fb0 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b2d092640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c920 tx=0x7f6b1800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18007c70 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b18007e10 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18018820 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b281a62a0 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.660+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a6830 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.661+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b28104290 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.662+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6b180189c0 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.663+0000 7f6b1effd640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 0x7f6b140a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.663+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f6b180ceb50 con 0x7f6b28076220 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.663+0000 7f6b2d893640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 0x7f6b140a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:18.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.663+0000 7f6b2d893640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 0x7f6b140a57b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b2402a910 tx=0x7f6b24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:54:18.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.666+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b18014030 con 0x7f6b28076220 2024-09-15T23:54:18.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:18.840+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6b28062630 con 0x7f6b140a32f0 2024-09-15T23:54:19.471 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:19.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:54:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T23:54:19.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:19.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:19.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:19.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:54:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T23:54:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:19.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:19.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:19.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:54:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:54:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] boot 2024-09-15T23:54:20.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T23:54:20.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] boot 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T23:54:20.456 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: osd.4 [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] boot 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T23:54:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T23:54:21.487 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:21 smithi102 ceph-mon[34458]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T23:54:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:21 smithi031 ceph-mon[31203]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T23:54:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:21 smithi081 ceph-mon[35060]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T23:54:22.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:22.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/1825963110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]': finished 2024-09-15T23:54:22.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T23:54:22.459 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:22.459 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:22 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/3448335960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1825963110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]': finished 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T23:54:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:22.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:22 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/3448335960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:22.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1825963110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]: dispatch 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010742fa-38fa-4483-9f2f-173c0fd11a06"}]': finished 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:22 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/3448335960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:24.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:24 smithi031 ceph-mon[31203]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:24 smithi081 ceph-mon[35060]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:24.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:24 smithi102 ceph-mon[34458]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:26.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:26 smithi102 ceph-mon[34458]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:26 smithi031 ceph-mon[31203]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:26.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:26 smithi081 ceph-mon[35060]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:28 smithi031 ceph-mon[31203]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:28.524 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:28 smithi102 ceph-mon[34458]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:28.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:28 smithi081 ceph-mon[35060]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:30.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:30 smithi102 ceph-mon[34458]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:30.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T23:54:30.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:30.468 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:30 smithi102 ceph-mon[34458]: Deploying daemon osd.5 on smithi102 2024-09-15T23:54:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:30 smithi031 ceph-mon[31203]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T23:54:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:30 smithi031 ceph-mon[31203]: Deploying daemon osd.5 on smithi102 2024-09-15T23:54:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:30 smithi081 ceph-mon[35060]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T23:54:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:30 smithi081 ceph-mon[35060]: Deploying daemon osd.5 on smithi102 2024-09-15T23:54:32.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:32 smithi031 ceph-mon[31203]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:32 smithi081 ceph-mon[35060]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:32.564 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:32 smithi102 ceph-mon[34458]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:34.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:34 smithi031 ceph-mon[31203]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:34.523 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:34 smithi102 ceph-mon[34458]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:34.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:34 smithi081 ceph-mon[35060]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:36 smithi031 ceph-mon[31203]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:36 smithi102 ceph-mon[34458]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:36.508 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:36 smithi081 ceph-mon[35060]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:38.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:38 smithi031 ceph-mon[31203]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:38.540 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:38 smithi102 ceph-mon[34458]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:38 smithi081 ceph-mon[35060]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:40.010 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.010 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.010 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:40.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:40.011 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.050+0000 7f6b1effd640 1 -- 172.21.15.102:0/1788209807 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b28062630 con 0x7f6b140a32f0 2024-09-15T23:54:40.051 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 5 on host 'smithi102' 2024-09-15T23:54:40.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 msgr2=0x7f6b140a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:40.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 0x7f6b140a57b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b2402a910 tx=0x7f6b24000bc0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 msgr2=0x7f6b2819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:40.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c920 tx=0x7f6b1800cde0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 shutdown_connections 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6b140a32f0 0x7f6b140a57b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b2810e1e0 0x7f6b281a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b28076220 0x7f6b2819e8c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 --2- 172.21.15.102:0/1788209807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b28074c50 0x7f6b2819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 >> 172.21.15.102:0/1788209807 conn(0x7f6b280fd390 msgr2=0x7f6b28101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 shutdown_connections 2024-09-15T23:54:40.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:54:40.052+0000 7f6b2faf7640 1 -- 172.21.15.102:0/1788209807 wait complete. 2024-09-15T23:54:40.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:40.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:40.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:40.750 DEBUG:teuthology.orchestra.run.smithi102:osd.5> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.5.service 2024-09-15T23:54:40.753 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-09-15T23:54:40.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T23:54:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:54:41.024 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:40 smithi102 ceph-mon[34458]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:41.024 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:41.025 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:41.025 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:41.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:40 smithi081 ceph-mon[35060]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:41.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:41.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:41.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:41.138 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:40 smithi031 ceph-mon[31203]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:41.138 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:54:41.139 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:41.139 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:42.371 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 15 23:54:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-15T23:54:42.115+0000 7f8207876640 -1 osd.5 0 log_to_monitors true 2024-09-15T23:54:42.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:42 smithi031 ceph-mon[31203]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:42.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:42 smithi031 ceph-mon[31203]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:42.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:42 smithi031 ceph-mon[31203]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:42 smithi081 ceph-mon[35060]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:42 smithi081 ceph-mon[35060]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:42 smithi081 ceph-mon[35060]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:43.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:42 smithi102 ceph-mon[34458]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:42 smithi102 ceph-mon[34458]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:42 smithi102 ceph-mon[34458]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T23:54:43.872 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:54:43.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:44.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:43 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-15T23:54:43.794+0000 7f82037f9640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T23:54:44.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-15T23:54:43.799+0000 7f81ff621640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:54:44.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T23:54:44.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T23:54:44.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:44.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:43 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.163 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:54:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T23:54:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T23:54:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:54:44.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:43 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 -- 172.21.15.31:0/3209587797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344102690 msgr2=0x7f0344108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 --2- 172.21.15.31:0/3209587797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344102690 0x7f0344108f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f033c009ca0 tx=0x7f033c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 -- 172.21.15.31:0/3209587797 shutdown_connections 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 --2- 172.21.15.31:0/3209587797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344102690 0x7f0344108f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 --2- 172.21.15.31:0/3209587797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344101cd0 0x7f0344102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 --2- 172.21.15.31:0/3209587797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0344100ad0 0x7f0344100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 -- 172.21.15.31:0/3209587797 >> 172.21.15.31:0/3209587797 conn(0x7f03440fc280 msgr2=0x7f03440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:54:44.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 -- 172.21.15.31:0/3209587797 shutdown_connections 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.658+0000 7f034b1f2640 1 -- 172.21.15.31:0/3209587797 wait complete. 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 Processor -- start 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 -- start start 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344100ad0 0x7f0344196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0344102690 0x7f034419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f034410c700 con 0x7f0344101cd0 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f034410c580 con 0x7f0344100ad0 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f034b1f2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f034410c880 con 0x7f0344102690 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47630/0 (socket says 172.21.15.31:47630) 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 -- 172.21.15.31:0/2574008123 learned_addr learned my addr 172.21.15.31:0/2574008123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:54:44.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0348f8e640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344100ad0 0x7f0344196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 -- 172.21.15.31:0/2574008123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0344102690 msgr2=0x7f034419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0344102690 0x7f034419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 -- 172.21.15.31:0/2574008123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344100ad0 msgr2=0x7f0344196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344100ad0 0x7f0344196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 -- 172.21.15.31:0/2574008123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034419e9e0 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.659+0000 7f0343fff640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f033000b4a0 tx=0x7f033000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.660+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330014070 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.660+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0330010ac0 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.660+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034419ecd0 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.660+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330017c30 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.660+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034419f280 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.661+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0344107f20 con 0x7f0344101cd0 2024-09-15T23:54:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.662+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0330017070 con 0x7f0344101cd0 2024-09-15T23:54:44.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.663+0000 7f0341ffb640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 0x7f03340a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:54:44.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.663+0000 7f0348f8e640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 0x7f03340a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:54:44.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.663+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7f03300cde60 con 0x7f0344101cd0 2024-09-15T23:54:44.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.663+0000 7f0348f8e640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 0x7f03340a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0344101b30 tx=0x7f0338008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:54:44.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.667+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f033009b980 con 0x7f0344101cd0 2024-09-15T23:54:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:54:44.852+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0344062630 con 0x7f03340a32f0 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi102 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] boot 2024-09-15T23:54:44.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T23:54:44.855 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi102 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] boot 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T23:54:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-15T23:54:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi102 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: osd.5 [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] boot 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T23:54:45.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T23:54:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:54:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:54:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:46.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:46.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:46.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:46.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:45 smithi081 ceph-mon[35060]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T23:54:46.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:54:46.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:54:46.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:45 smithi031 ceph-mon[31203]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T23:54:46.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:54:46.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:54:46.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:54:46.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:54:46.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:54:46.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:46.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:45 smithi102 ceph-mon[34458]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T23:54:47.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:46 smithi031 ceph-mon[31203]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:47.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:46 smithi031 ceph-mon[31203]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T23:54:47.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:46 smithi081 ceph-mon[35060]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:47.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:46 smithi081 ceph-mon[35060]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T23:54:47.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:46 smithi102 ceph-mon[34458]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:47.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:46 smithi102 ceph-mon[34458]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T23:54:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:47 smithi031 ceph-mon[31203]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T23:54:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:47 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]: dispatch 2024-09-15T23:54:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:47 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]': finished 2024-09-15T23:54:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:47 smithi031 ceph-mon[31203]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T23:54:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:54:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:47 smithi081 ceph-mon[35060]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T23:54:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:47 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]: dispatch 2024-09-15T23:54:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:47 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]': finished 2024-09-15T23:54:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:47 smithi081 ceph-mon[35060]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T23:54:48.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:54:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:47 smithi102 ceph-mon[34458]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T23:54:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:47 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]: dispatch 2024-09-15T23:54:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:47 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1046407746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7c78c23-60ea-4bb6-b269-9d236295e6b4"}]': finished 2024-09-15T23:54:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:47 smithi102 ceph-mon[34458]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T23:54:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:54:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:48 smithi031 ceph-mon[31203]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:48 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/350107766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:49.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:48 smithi081 ceph-mon[35060]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:49.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:48 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/350107766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:48 smithi102 ceph-mon[34458]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:54:49.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:48 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/350107766' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:54:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:50 smithi031 ceph-mon[31203]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-15T23:54:51.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:50 smithi081 ceph-mon[35060]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-15T23:54:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:50 smithi102 ceph-mon[34458]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-15T23:54:53.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:52 smithi031 ceph-mon[31203]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:54:53.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:52 smithi102 ceph-mon[34458]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:54:53.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:52 smithi081 ceph-mon[35060]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:54:55.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:54 smithi031 ceph-mon[31203]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T23:54:55.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:54 smithi081 ceph-mon[35060]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T23:54:55.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:54 smithi102 ceph-mon[34458]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T23:54:57.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:56 smithi031 ceph-mon[31203]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-15T23:54:57.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T23:54:57.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:57.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:56 smithi031 ceph-mon[31203]: Deploying daemon osd.6 on smithi031 2024-09-15T23:54:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:56 smithi081 ceph-mon[35060]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-15T23:54:57.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T23:54:57.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:57.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:56 smithi081 ceph-mon[35060]: Deploying daemon osd.6 on smithi031 2024-09-15T23:54:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:56 smithi102 ceph-mon[34458]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-15T23:54:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T23:54:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:54:57.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:56 smithi102 ceph-mon[34458]: Deploying daemon osd.6 on smithi031 2024-09-15T23:54:59.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:54:58 smithi031 ceph-mon[31203]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T23:54:59.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:54:58 smithi081 ceph-mon[35060]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T23:54:59.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:54:58 smithi102 ceph-mon[34458]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T23:55:01.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:00 smithi031 ceph-mon[31203]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T23:55:01.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:00 smithi081 ceph-mon[35060]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T23:55:01.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:00 smithi102 ceph-mon[34458]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T23:55:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:02 smithi031 ceph-mon[31203]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:02 smithi081 ceph-mon[35060]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:03.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:02 smithi102 ceph-mon[34458]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:03.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:04 smithi031 ceph-mon[31203]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:04 smithi081 ceph-mon[35060]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:04 smithi102 ceph-mon[34458]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:05.790 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-09-15T23:55:05.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.789+0000 7f0341ffb640 1 -- 172.21.15.31:0/2574008123 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0344062630 con 0x7f03340a32f0 2024-09-15T23:55:05.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 msgr2=0x7f03340a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:05.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 0x7f03340a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0344101b30 tx=0x7f0338008040 comp rx=0 tx=0).stop 2024-09-15T23:55:05.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 msgr2=0x7f03441972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:05.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f033000b4a0 tx=0x7f033000b960 comp rx=0 tx=0).stop 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 shutdown_connections 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f03340a32f0 0x7f03340a57b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0344102690 0x7f034419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0344101cd0 0x7f03441972f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 --2- 172.21.15.31:0/2574008123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0344100ad0 0x7f0344196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 >> 172.21.15.31:0/2574008123 conn(0x7f03440fc280 msgr2=0x7f03441050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.791+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 shutdown_connections 2024-09-15T23:55:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:55:05.792+0000 7f034b1f2640 1 -- 172.21.15.31:0/2574008123 wait complete. 2024-09-15T23:55:06.516 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.6.service 2024-09-15T23:55:06.519 INFO:tasks.cephadm:Deploying osd.7 on smithi081 with /dev/vg_nvme/lv_2... 2024-09-15T23:55:06.519 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T23:55:06.774 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:55:06.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:06 smithi081 ceph-mon[35060]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:06.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:06.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:06.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:06.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:06 smithi031 ceph-mon[31203]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:06 smithi102 ceph-mon[34458]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:07.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:07.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:08.431 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 15 23:55:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-15T23:55:08.244+0000 7fcd09f3e640 -1 osd.6 0 log_to_monitors true 2024-09-15T23:55:09.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:08 smithi081 ceph-mon[35060]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:09.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:08 smithi081 ceph-mon[35060]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T23:55:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:08 smithi102 ceph-mon[34458]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:08 smithi102 ceph-mon[34458]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T23:55:09.223 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:08 smithi031 ceph-mon[31203]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:09.223 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:08 smithi031 ceph-mon[31203]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T23:55:09.451 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:55:09.480 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2024-09-15T23:55:09.732 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:55:09.923 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-15T23:55:09.795+0000 7fcd05ec1640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T23:55:09.923 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-15T23:55:09.804+0000 7fcd01ce9640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:55:09.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:09 smithi031 ceph-mon[31203]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T23:55:10.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T23:55:10.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T23:55:10.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:55:10.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:10.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:55:10.012 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:09 smithi081 ceph-mon[35060]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: from='osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:55:10.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:09 smithi102 ceph-mon[34458]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T23:55:10.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 -- 172.21.15.81:0/959543202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40201037c0 msgr2=0x7f402010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:10.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 --2- 172.21.15.81:0/959543202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40201037c0 0x7f402010a050 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4018009ca0 tx=0x7f401802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 -- 172.21.15.81:0/959543202 shutdown_connections 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 --2- 172.21.15.81:0/959543202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40201037c0 0x7f402010a050 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 --2- 172.21.15.81:0/959543202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4020102e00 0x7f4020103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 --2- 172.21.15.81:0/959543202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 -- 172.21.15.81:0/959543202 >> 172.21.15.81:0/959543202 conn(0x7f40200fd3b0 msgr2=0x7f40200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 -- 172.21.15.81:0/959543202 shutdown_connections 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.236+0000 7f402793c640 1 -- 172.21.15.81:0/959543202 wait complete. 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 Processor -- start 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 -- start start 2024-09-15T23:55:10.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 0x7f402019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f402010d790 con 0x7f4020101c00 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f402010d610 con 0x7f4020102e00 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f402793c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f402010d910 con 0x7f40201037c0 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f40256d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59054/0 (socket says 172.21.15.81:59054) 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f40256d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.81:58998/0 (socket says 172.21.15.81:58998) 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 -- 172.21.15.81:0/804017483 learned_addr learned my addr 172.21.15.81:0/804017483 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-15T23:55:10.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4025ed9640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 0x7f402019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 -- 172.21.15.81:0/804017483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 msgr2=0x7f402019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 0x7f402019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 -- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 msgr2=0x7f4020197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4024ed7640 1 -- 172.21.15.81:0/804017483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402019fa10 con 0x7f4020102e00 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f4025ed9640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 0x7f402019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:55:10.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.237+0000 7f40256d8640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f4024ed7640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f401c00b4a0 tx=0x7f401c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c014070 con 0x7f4020102e00 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c010ac0 con 0x7f4020102e00 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402019fd00 con 0x7f4020102e00 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c017c30 con 0x7f4020102e00 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.238+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40201a0190 con 0x7f4020102e00 2024-09-15T23:55:10.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.239+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4020108f60 con 0x7f4020102e00 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.240+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f401c017070 con 0x7f4020102e00 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.241+0000 7f40167fc640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 0x7f400c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.241+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7f401c0ce290 con 0x7f4020102e00 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.241+0000 7f40256d8640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 0x7f400c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.241+0000 7f40256d8640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 0x7f400c0a5730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4020102c60 tx=0x7f4010005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:55:10.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.248+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f401c09bdc0 con 0x7f4020102e00 2024-09-15T23:55:10.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:10.429+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4020062630 con 0x7f400c0a3270 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi031 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:11.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:11.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:10 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi031 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:10 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:11.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi031 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:11.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:10 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] boot 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T23:55:11.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:11 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:12.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:55:12.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:55:12.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:12.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] boot 2024-09-15T23:55:12.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T23:55:12.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:11 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:12.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:55:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:55:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: osd.6 [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] boot 2024-09-15T23:55:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T23:55:12.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:11 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T23:55:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:12 smithi081 ceph-mon[35060]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:12 smithi081 ceph-mon[35060]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T23:55:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:12 smithi102 ceph-mon[34458]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:12 smithi102 ceph-mon[34458]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T23:55:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:12 smithi031 ceph-mon[31203]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-15T23:55:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:12 smithi031 ceph-mon[31203]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T23:55:14.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/2115798847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]': finished 2024-09-15T23:55:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T23:55:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:14.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/3589827511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:14.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/2115798847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]': finished 2024-09-15T23:55:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T23:55:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:14.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/3589827511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/2115798847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]: dispatch 2024-09-15T23:55:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f270c537-d25a-4ee7-87a0-b18f67a00a2c"}]': finished 2024-09-15T23:55:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T23:55:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/3589827511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:15.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:15 smithi081 ceph-mon[35060]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:15 smithi102 ceph-mon[34458]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:15 smithi031 ceph-mon[31203]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:17 smithi102 ceph-mon[34458]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:17 smithi031 ceph-mon[31203]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:17.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:17 smithi081 ceph-mon[35060]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:19.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:19 smithi081 ceph-mon[35060]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:19.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:19 smithi102 ceph-mon[34458]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:19 smithi031 ceph-mon[31203]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:21.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:21 smithi081 ceph-mon[35060]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:21 smithi102 ceph-mon[34458]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:21 smithi031 ceph-mon[31203]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:22.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T23:55:22.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T23:55:22.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T23:55:22.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:23 smithi102 ceph-mon[34458]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:23 smithi102 ceph-mon[34458]: Deploying daemon osd.7 on smithi081 2024-09-15T23:55:23.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:23 smithi081 ceph-mon[35060]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:23.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:23 smithi081 ceph-mon[35060]: Deploying daemon osd.7 on smithi081 2024-09-15T23:55:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:23 smithi031 ceph-mon[31203]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:23 smithi031 ceph-mon[31203]: Deploying daemon osd.7 on smithi081 2024-09-15T23:55:25.221 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:25 smithi081 ceph-mon[35060]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:25 smithi102 ceph-mon[34458]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:25 smithi031 ceph-mon[31203]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:27.185 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:27 smithi081 ceph-mon[35060]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:27 smithi102 ceph-mon[34458]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:27 smithi031 ceph-mon[31203]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:28.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:28 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:28.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:28 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:28.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:28 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:28.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:28 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:28.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:28 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:28.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:28 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:28 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:28 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:28 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:29.338 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:29 smithi081 ceph-mon[35060]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:29 smithi102 ceph-mon[34458]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:29.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:29 smithi031 ceph-mon[31203]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:31.250 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:31 smithi081 ceph-mon[35060]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:31.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.286+0000 7f40167fc640 1 -- 172.21.15.81:0/804017483 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4020062630 con 0x7f400c0a3270 2024-09-15T23:55:31.287 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 7 on host 'smithi081' 2024-09-15T23:55:31.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 msgr2=0x7f400c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 0x7f400c0a5730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4020102c60 tx=0x7f4010005f90 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 msgr2=0x7f4020198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f401c00b4a0 tx=0x7f401c00b960 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 shutdown_connections 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f400c0a3270 0x7f400c0a5730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40201037c0 0x7f402019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4020102e00 0x7f4020198320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 --2- 172.21.15.81:0/804017483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4020101c00 0x7f4020197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.288+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 >> 172.21.15.81:0/804017483 conn(0x7f40200fd3b0 msgr2=0x7f4020106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.289+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 shutdown_connections 2024-09-15T23:55:31.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:55:31.289+0000 7f402793c640 1 -- 172.21.15.81:0/804017483 wait complete. 2024-09-15T23:55:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:31 smithi102 ceph-mon[34458]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:31 smithi031 ceph-mon[31203]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:32.020 DEBUG:teuthology.orchestra.run.smithi081:osd.7> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.7.service 2024-09-15T23:55:32.022 INFO:tasks.cephadm:Deploying osd.8 on smithi102 with /dev/vg_nvme/lv_2... 2024-09-15T23:55:32.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T23:55:32.274 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:55:32.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:32 smithi102 ceph-mon[34458]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:32.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:32 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:32 smithi081 ceph-mon[35060]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:32 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:32 smithi031 ceph-mon[31203]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:32.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:32.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:32 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:34.033 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 15 23:55:33 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-15T23:55:33.858+0000 7fa618cf5640 -1 osd.7 0 log_to_monitors true 2024-09-15T23:55:34.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:34 smithi102 ceph-mon[34458]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:34.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:34 smithi102 ceph-mon[34458]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:34 smithi102 ceph-mon[34458]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:34 smithi081 ceph-mon[35060]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:34.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:34 smithi081 ceph-mon[35060]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:34 smithi081 ceph-mon[35060]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:34 smithi031 ceph-mon[31203]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:34.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:34 smithi031 ceph-mon[31203]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:34 smithi031 ceph-mon[31203]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T23:55:34.832 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:55:34.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-09-15T23:55:35.107 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:55:35.499 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:35 smithi102 ceph-mon[34458]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T23:55:35.499 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:35 smithi102 ceph-mon[34458]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.499 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:35 smithi102 ceph-mon[34458]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T23:55:35.499 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:35 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:35.499 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:35 smithi102 ceph-mon[34458]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-mon[35060]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T23:55:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-mon[35060]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-mon[35060]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T23:55:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-mon[35060]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 -- 172.21.15.102:0/4067902725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8101c00 msgr2=0x7fadc8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 --2- 172.21.15.102:0/4067902725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8101c00 0x7fadc8102000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009900 tx=0x7fadb802a410 comp rx=0 tx=0).stop 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 -- 172.21.15.102:0/4067902725 shutdown_connections 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 --2- 172.21.15.102:0/4067902725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc81037c0 0x7fadc810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 --2- 172.21.15.102:0/4067902725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 0x7fadc8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 --2- 172.21.15.102:0/4067902725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8101c00 0x7fadc8102000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.583+0000 7fadce35f640 1 -- 172.21.15.102:0/4067902725 >> 172.21.15.102:0/4067902725 conn(0x7fadc80fd390 msgr2=0x7fadc80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- 172.21.15.102:0/4067902725 shutdown_connections 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- 172.21.15.102:0/4067902725 wait complete. 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 Processor -- start 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- start start 2024-09-15T23:55:35.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc8101c00 0x7fadc8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 0x7fadc80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fadc810d710 con 0x7fadc8101c00 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fadc810d590 con 0x7fadc8102e00 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadce35f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fadc810d890 con 0x7fadc81037c0 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.584+0000 7fadcc8fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47882/0 (socket says 172.21.15.102:47882) 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 -- 172.21.15.102:0/293853225 learned_addr learned my addr 172.21.15.102:0/293853225 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadc7fff640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc8101c00 0x7fadc8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadc77fe640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 0x7fadc80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 -- 172.21.15.102:0/293853225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 msgr2=0x7fadc80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:35.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 0x7fadc80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 -- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc8101c00 msgr2=0x7fadc8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc8101c00 0x7fadc8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 -- 172.21.15.102:0/293853225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc80753e0 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadcc8fc640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadc000eaa0 tx=0x7fadc000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000cdb0 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadc0005020 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00116d0 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc81a1fc0 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.585+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81a2470 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.586+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8109140 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.587+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fadc00051d0 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.588+0000 7fadc57fa640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 0x7fadb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.588+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7fadc00ced70 con 0x7fadc81037c0 2024-09-15T23:55:35.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.588+0000 7fadc7fff640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 0x7fadb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:55:35.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.589+0000 7fadc7fff640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 0x7fadb00a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadb8004770 tx=0x7fadb8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:55:35.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.592+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fadc0016030 con 0x7fadc81037c0 2024-09-15T23:55:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:35 smithi031 ceph-mon[31203]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T23:55:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:35 smithi031 ceph-mon[31203]: from='osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:35 smithi031 ceph-mon[31203]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T23:55:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:35 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:35 smithi031 ceph-mon[31203]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:55:35.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:35.771+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fadc8062630 con 0x7fadb00a3240 2024-09-15T23:55:36.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-15T23:55:35.732+0000 7fa61548c640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T23:55:36.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 15 23:55:35 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-15T23:55:35.740+0000 7fa60ca98640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:55:36.436 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi081 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.437 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='osd.7 ' entity='osd.7' 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:36.438 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:36.439 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:55:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T23:55:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi081 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-15T23:55:36.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='osd.7 ' entity='osd.7' 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:36.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-15T23:55:36.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:55:36.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi081 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 5696M 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='osd.7 ' entity='osd.7' 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:55:36.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:37.477 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:37 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:55:37.477 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:37 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:55:37.477 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:37 smithi102 ceph-mon[34458]: osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] boot 2024-09-15T23:55:37.477 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:37 smithi102 ceph-mon[34458]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T23:55:37.477 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:37 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:37 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:55:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:37 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:55:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:37 smithi031 ceph-mon[31203]: osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] boot 2024-09-15T23:55:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:37 smithi031 ceph-mon[31203]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T23:55:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:37 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:37.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:37 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:55:37.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:37 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:55:37.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:37 smithi081 ceph-mon[35060]: osd.7 [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] boot 2024-09-15T23:55:37.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:37 smithi081 ceph-mon[35060]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T23:55:37.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:37 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T23:55:38.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1905538968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]': finished 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T23:55:38.625 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:38.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:38.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T23:55:38.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1905538968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]': finished 2024-09-15T23:55:38.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T23:55:38.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/1905538968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]: dispatch 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530bcf2d-1d5c-4ad2-bba2-fad10d3e581e"}]': finished 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T23:55:38.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:39 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/4247839053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:39.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:39 smithi102 ceph-mon[34458]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T23:55:39.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:39 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:39 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/4247839053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:39 smithi031 ceph-mon[31203]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T23:55:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:39 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:39.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:39 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/4247839053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:55:39.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:39 smithi081 ceph-mon[35060]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T23:55:39.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:39 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:40 smithi031 ceph-mon[31203]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:40 smithi031 ceph-mon[31203]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T23:55:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:41.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:40 smithi081 ceph-mon[35060]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:41.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:40 smithi081 ceph-mon[35060]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T23:55:41.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:40 smithi102 ceph-mon[34458]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:40 smithi102 ceph-mon[34458]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T23:55:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:55:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:42 smithi031 ceph-mon[31203]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:43.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:42 smithi081 ceph-mon[35060]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:43.325 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:42 smithi102 ceph-mon[34458]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:44 smithi102 ceph-mon[34458]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:44 smithi031 ceph-mon[31203]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:45.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:44 smithi081 ceph-mon[35060]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:47.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:46 smithi102 ceph-mon[34458]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:47.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T23:55:47.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:46 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:47.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:46 smithi031 ceph-mon[31203]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:47.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:46 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T23:55:47.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:46 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:47.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:46 smithi081 ceph-mon[35060]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:47.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:46 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T23:55:47.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:46 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:48.207 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:47 smithi102 ceph-mon[34458]: Deploying daemon osd.8 on smithi102 2024-09-15T23:55:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:47 smithi031 ceph-mon[31203]: Deploying daemon osd.8 on smithi102 2024-09-15T23:55:48.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:47 smithi081 ceph-mon[35060]: Deploying daemon osd.8 on smithi102 2024-09-15T23:55:49.247 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:48 smithi102 ceph-mon[34458]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:48 smithi031 ceph-mon[31203]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:49.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:48 smithi081 ceph-mon[35060]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:50 smithi031 ceph-mon[31203]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:51.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:50 smithi081 ceph-mon[35060]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:50 smithi102 ceph-mon[34458]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:53.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:52 smithi102 ceph-mon[34458]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:53.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:53.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:53.188 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:52 smithi031 ceph-mon[31203]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:53.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:52 smithi081 ceph-mon[35060]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:54 smithi031 ceph-mon[31203]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:55.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:54 smithi081 ceph-mon[35060]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:55.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:54 smithi102 ceph-mon[34458]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:56.246 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 8 on host 'smithi102' 2024-09-15T23:55:56.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.245+0000 7fadc57fa640 1 -- 172.21.15.102:0/293853225 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fadc8062630 con 0x7fadb00a3240 2024-09-15T23:55:56.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.247+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 msgr2=0x7fadb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:56.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.247+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 0x7fadb00a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadb8004770 tx=0x7fadb8000bc0 comp rx=0 tx=0).stop 2024-09-15T23:55:56.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 msgr2=0x7fadc8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:55:56.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadc000eaa0 tx=0x7fadc000ef60 comp rx=0 tx=0).stop 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 shutdown_connections 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadb00a3240 0x7fadb00a5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81037c0 0x7fadc8074b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadc8102e00 0x7fadc80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 --2- 172.21.15.102:0/293853225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadc8101c00 0x7fadc8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 >> 172.21.15.102:0/293853225 conn(0x7fadc80fd390 msgr2=0x7fadc8106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 shutdown_connections 2024-09-15T23:55:56.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:55:56.248+0000 7fadce35f640 1 -- 172.21.15.102:0/293853225 wait complete. 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:55:56.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:55:56.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:55:56.880 DEBUG:teuthology.orchestra.run.smithi102:osd.8> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.8.service 2024-09-15T23:55:56.882 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-09-15T23:55:56.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T23:55:57.149 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:55:58.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:58 smithi102 ceph-mon[34458]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:58.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:58 smithi081 ceph-mon[35060]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:58.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:58 smithi031 ceph-mon[31203]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:55:58.681 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 15 23:55:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-15T23:55:58.391+0000 7f081962d640 -1 osd.8 0 log_to_monitors true 2024-09-15T23:55:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:59 smithi031 ceph-mon[31203]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:55:59 smithi031 ceph-mon[31203]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:59 smithi102 ceph-mon[34458]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:55:59 smithi102 ceph-mon[34458]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:59 smithi081 ceph-mon[35060]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:55:59 smithi081 ceph-mon[35060]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T23:55:59.883 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:55:59.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-09-15T23:56:00.179 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi102 2024-09-15T23:56:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:00.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:00 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:00.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T23:56:00.588 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi102 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:00.589 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:00 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-15T23:56:00.291+0000 7f08155b0640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T23:56:00.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-15T23:56:00.299+0000 7f08113d8640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:56:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi102 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:00.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:00 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:00.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 -- 172.21.15.31:0/1132174534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 msgr2=0x7fadec1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 --2- 172.21.15.31:0/1132174534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec1073e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faddc009900 tx=0x7faddc02a410 comp rx=0 tx=0).stop 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 -- 172.21.15.31:0/1132174534 shutdown_connections 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 --2- 172.21.15.31:0/1132174534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec107920 0x7fadec109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 --2- 172.21.15.31:0/1132174534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec1073e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 --2- 172.21.15.31:0/1132174534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec1026c0 0x7fadec104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 -- 172.21.15.31:0/1132174534 >> 172.21.15.31:0/1132174534 conn(0x7fadec0fc280 msgr2=0x7fadec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 -- 172.21.15.31:0/1132174534 shutdown_connections 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.685+0000 7fadf4056640 1 -- 172.21.15.31:0/1132174534 wait complete. 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 Processor -- start 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 -- start start 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec1026c0 0x7fadec197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:00.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec107920 0x7fadec19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10d750 con 0x7fadec104ff0 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10d5d0 con 0x7fadec107920 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf4056640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fadec10d8d0 con 0x7fadec1026c0 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39860/0 (socket says 172.21.15.31:39860) 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 -- 172.21.15.31:0/4269258441 learned_addr learned my addr 172.21.15.31:0/4269258441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf1df2640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec1026c0 0x7fadec197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf25f3640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec107920 0x7fadec19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 -- 172.21.15.31:0/4269258441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec1026c0 msgr2=0x7fadec197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec1026c0 0x7fadec197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 -- 172.21.15.31:0/4269258441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec107920 msgr2=0x7fadec19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec107920 0x7fadec19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.686+0000 7fadf15f1640 1 -- 172.21.15.31:0/4269258441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec19fa70 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fadf15f1640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faddc0314e0 tx=0x7faddc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc03f070 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faddc031e40 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec19fd00 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc035030 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.687+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec1a0190 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.688+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadec0fdd70 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.689+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7faddc042070 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.690+0000 7fade2ffd640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 0x7fadd80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.690+0000 7fadf1df2640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 0x7fadd80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.690+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5549+0+0 (secure 0 0 0) 0x7faddc0efbd0 con 0x7fadec104ff0 2024-09-15T23:56:00.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.690+0000 7fadf1df2640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 0x7fadd80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fade800bbc0 tx=0x7fade800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.693+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faddc0bccd0 con 0x7fadec104ff0 2024-09-15T23:56:00.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:00.878+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fadec062630 con 0x7fadd80a3270 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:01 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:56:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:01 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:01.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:01 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] boot 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T23:56:02.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:02.580 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] boot 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T23:56:02.581 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:02.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: osd.8 [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] boot 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T23:56:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T23:56:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:03 smithi031 ceph-mon[31203]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T23:56:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:03 smithi102 ceph-mon[34458]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T23:56:03.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:03 smithi081 ceph-mon[35060]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]: dispatch 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]': finished 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:04.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2800583251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:04.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:04.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T23:56:04.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]: dispatch 2024-09-15T23:56:04.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]': finished 2024-09-15T23:56:04.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T23:56:04.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:04.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2800583251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:04.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]: dispatch 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3587306311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1bf1827-c5c0-4b3b-9f97-f7eb1679d742"}]': finished 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:04.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2800583251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:05.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:05 smithi031 ceph-mon[31203]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T23:56:05.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:05.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:05 smithi081 ceph-mon[35060]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T23:56:05.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:05.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:05 smithi102 ceph-mon[34458]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T23:56:05.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:06.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:06 smithi031 ceph-mon[31203]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:06.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:06 smithi081 ceph-mon[35060]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:06.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:06 smithi102 ceph-mon[34458]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:08.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:08 smithi081 ceph-mon[35060]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-15T23:56:08.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:08 smithi102 ceph-mon[34458]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-15T23:56:08.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:08 smithi031 ceph-mon[31203]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-15T23:56:10.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:10 smithi031 ceph-mon[31203]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:56:10.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:10 smithi081 ceph-mon[35060]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:56:10.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:10 smithi102 ceph-mon[34458]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T23:56:12.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:12 smithi031 ceph-mon[31203]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T23:56:12.553 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T23:56:12.553 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:12.553 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:12 smithi031 ceph-mon[31203]: Deploying daemon osd.9 on smithi031 2024-09-15T23:56:12.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:12 smithi081 ceph-mon[35060]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T23:56:12.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T23:56:12.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:12.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:12 smithi081 ceph-mon[35060]: Deploying daemon osd.9 on smithi031 2024-09-15T23:56:12.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:12 smithi102 ceph-mon[34458]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T23:56:12.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T23:56:12.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:12.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:12 smithi102 ceph-mon[34458]: Deploying daemon osd.9 on smithi031 2024-09-15T23:56:14.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:14 smithi081 ceph-mon[35060]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-15T23:56:14.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:14 smithi102 ceph-mon[34458]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-15T23:56:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:14 smithi031 ceph-mon[31203]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-15T23:56:16.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:16 smithi081 ceph-mon[35060]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T23:56:16.873 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:16 smithi102 ceph-mon[34458]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T23:56:16.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:16 smithi031 ceph-mon[31203]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T23:56:18.778 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:18 smithi031 ceph-mon[31203]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:56:18.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:18.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:18.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:18.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:18 smithi081 ceph-mon[35060]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:56:18.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:18.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:18.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:18.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:18 smithi102 ceph-mon[34458]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T23:56:18.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:18.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:18.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:20.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:20 smithi081 ceph-mon[35060]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:20.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:20 smithi031 ceph-mon[31203]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:20.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:20 smithi102 ceph-mon[34458]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.028+0000 7fade2ffd640 1 -- 172.21.15.31:0/4269258441 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fadec062630 con 0x7fadd80a3270 2024-09-15T23:56:22.029 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 msgr2=0x7fadd80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 0x7fadd80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fade800bbc0 tx=0x7fade800b390 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 msgr2=0x7fadec198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faddc0314e0 tx=0x7faddc003c80 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 shutdown_connections 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fadd80a3270 0x7fadd80a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fadec107920 0x7fadec19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadec104ff0 0x7fadec198380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 --2- 172.21.15.31:0/4269258441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadec1026c0 0x7fadec197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 >> 172.21.15.31:0/4269258441 conn(0x7fadec0fc280 msgr2=0x7fadec103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:56:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 shutdown_connections 2024-09-15T23:56:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:56:22.030+0000 7fadf4056640 1 -- 172.21.15.31:0/4269258441 wait complete. 2024-09-15T23:56:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:22 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:22 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:22.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:22 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:22.801 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.9.service 2024-09-15T23:56:22.803 INFO:tasks.cephadm:Deploying osd.10 on smithi081 with /dev/vg_nvme/lv_1... 2024-09-15T23:56:22.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T23:56:23.059 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:56:24.205 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 15 23:56:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-15T23:56:23.906+0000 7f53b3329640 -1 osd.9 0 log_to_monitors true 2024-09-15T23:56:24.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:24 smithi031 ceph-mon[31203]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:24.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:24 smithi031 ceph-mon[31203]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T23:56:24.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:24 smithi081 ceph-mon[35060]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:24.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:24 smithi081 ceph-mon[35060]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T23:56:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:24 smithi102 ceph-mon[34458]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:24 smithi102 ceph-mon[34458]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T23:56:25.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:25 smithi081 ceph-mon[35060]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T23:56:25.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:25 smithi081 ceph-mon[35060]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T23:56:25.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:25 smithi081 ceph-mon[35060]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:56:25.390 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:25 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:25.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-mon[31203]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T23:56:25.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-mon[31203]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T23:56:25.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-mon[31203]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:56:25.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:25.431 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-15T23:56:25.216+0000 7f53af2ac640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T23:56:25.431 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 15 23:56:25 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-15T23:56:25.226+0000 7f53ab0d4640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:56:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:25 smithi102 ceph-mon[34458]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T23:56:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:25 smithi102 ceph-mon[34458]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T23:56:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:25 smithi102 ceph-mon[34458]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-15T23:56:25.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:25 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:25.701 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-15T23:56:25.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_1 2024-09-15T23:56:25.975 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:56:26.427 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi031 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.428 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-15T23:56:26.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:26.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:26.429 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:56:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T23:56:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi031 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-15T23:56:26.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:26.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:26.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 -- 172.21.15.81:0/3320787868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 msgr2=0x7f9d14102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:26.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 --2- 172.21.15.81:0/3320787868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14102000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009900 tx=0x7f9d0402a410 comp rx=0 tx=0).stop 2024-09-15T23:56:26.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 -- 172.21.15.81:0/3320787868 shutdown_connections 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 --2- 172.21.15.81:0/3320787868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 0x7f9d1410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 --2- 172.21.15.81:0/3320787868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 0x7f9d14103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 --2- 172.21.15.81:0/3320787868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14102000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 -- 172.21.15.81:0/3320787868 >> 172.21.15.81:0/3320787868 conn(0x7f9d140fd390 msgr2=0x7f9d140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 -- 172.21.15.81:0/3320787868 shutdown_connections 2024-09-15T23:56:26.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.510+0000 7f9d19a15640 1 -- 172.21.15.81:0/3320787868 wait complete. 2024-09-15T23:56:26.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 Processor -- start 2024-09-15T23:56:26.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 -- start start 2024-09-15T23:56:26.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:26.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 0x7f9d141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:26.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 0x7f9d1419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:26.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1410d730 con 0x7f9d141037c0 2024-09-15T23:56:26.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1410d5b0 con 0x7f9d14101c00 2024-09-15T23:56:26.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d19a15640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1410d8b0 con 0x7f9d14102e00 2024-09-15T23:56:26.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d12ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:26.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d12ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55174/0 (socket says 172.21.15.81:55174) 2024-09-15T23:56:26.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d12ffd640 1 -- 172.21.15.81:0/1400718160 learned_addr learned my addr 172.21.15.81:0/1400718160 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-09-15T23:56:26.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.511+0000 7f9d0a7fc640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 0x7f9d141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:26.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d137fe640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 0x7f9d1419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:26.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 -- 172.21.15.81:0/1400718160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 msgr2=0x7f9d141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:26.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 0x7f9d141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 -- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 msgr2=0x7f9d1419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 0x7f9d1419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 -- 172.21.15.81:0/1400718160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1419fa90 con 0x7f9d14101c00 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d12ffd640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d040314e0 tx=0x7f9d04031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0403f030 con 0x7f9d14101c00 2024-09-15T23:56:26.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d040044b0 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1419fd20 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04035070 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.512+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a01b0 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.513+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d141090c0 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.514+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9d04042030 con 0x7f9d14101c00 2024-09-15T23:56:26.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.515+0000 7f9d18a13640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 0x7f9d000a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:26.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.516+0000 7f9d0a7fc640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 0x7f9d000a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:26.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.516+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f9d040efac0 con 0x7f9d14101c00 2024-09-15T23:56:26.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.516+0000 7f9d0a7fc640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 0x7f9d000a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d14199380 tx=0x7f9cfc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:26.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.519+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d040bcf40 con 0x7f9d14101c00 2024-09-15T23:56:26.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:26.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-15T23:56:26.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T23:56:26.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi031 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:26.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:26.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:26.694+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9d14062630 con 0x7f9d000a3270 2024-09-15T23:56:27.364 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:56:27.364 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] boot 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:27.365 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:27 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] boot 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T23:56:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:27.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:27.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:27.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:27 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: osd.9 [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] boot 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:27.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:27 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:28 smithi031 ceph-mon[31203]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:28 smithi031 ceph-mon[31203]: osdmap e63: 10 total, 10 up, 10 in 2024-09-15T23:56:28.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:28 smithi081 ceph-mon[35060]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:28.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:28 smithi081 ceph-mon[35060]: osdmap e63: 10 total, 10 up, 10 in 2024-09-15T23:56:28.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:28 smithi102 ceph-mon[34458]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-15T23:56:28.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:28 smithi102 ceph-mon[34458]: osdmap e63: 10 total, 10 up, 10 in 2024-09-15T23:56:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/2337710752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]': finished 2024-09-15T23:56:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T23:56:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:30 smithi031 ceph-mon[31203]: from='client.? 172.21.15.81:0/2040661263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/2337710752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]': finished 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:30 smithi081 ceph-mon[35060]: from='client.? 172.21.15.81:0/2040661263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:30.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:30.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/2337710752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]: dispatch 2024-09-15T23:56:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a086c66d-46cc-4b3a-a8cf-7022a6acf89e"}]': finished 2024-09-15T23:56:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T23:56:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:30 smithi102 ceph-mon[34458]: from='client.? 172.21.15.81:0/2040661263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:32.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:32 smithi031 ceph-mon[31203]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:32.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:32 smithi081 ceph-mon[35060]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:32.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:32 smithi102 ceph-mon[34458]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:34.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:34 smithi031 ceph-mon[31203]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:34.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:34 smithi081 ceph-mon[35060]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:34.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:34 smithi102 ceph-mon[34458]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:36.489 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:36 smithi081 ceph-mon[35060]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:36 smithi031 ceph-mon[31203]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:36.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:36 smithi102 ceph-mon[34458]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:38.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:38 smithi031 ceph-mon[31203]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:38.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T23:56:38.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:38 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:38.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:38 smithi031 ceph-mon[31203]: Deploying daemon osd.10 on smithi081 2024-09-15T23:56:38.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:38 smithi081 ceph-mon[35060]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T23:56:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:38 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:38 smithi081 ceph-mon[35060]: Deploying daemon osd.10 on smithi081 2024-09-15T23:56:38.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:38 smithi102 ceph-mon[34458]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:38.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T23:56:38.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:38 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:38.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:38 smithi102 ceph-mon[34458]: Deploying daemon osd.10 on smithi081 2024-09-15T23:56:40.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:40 smithi031 ceph-mon[31203]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:40.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:40 smithi081 ceph-mon[35060]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:40.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:40 smithi102 ceph-mon[34458]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:42.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:42 smithi081 ceph-mon[35060]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:42.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:42 smithi102 ceph-mon[34458]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:42.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:42 smithi031 ceph-mon[31203]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:44.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:44 smithi081 ceph-mon[35060]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:44.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:44.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:44.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:44.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:44 smithi031 ceph-mon[31203]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:44.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:44.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:44.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:44.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:44 smithi102 ceph-mon[34458]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:44.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:46.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:46 smithi081 ceph-mon[35060]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:46.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:46 smithi102 ceph-mon[34458]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:46.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:46 smithi031 ceph-mon[31203]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:47.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.571+0000 7f9d18a13640 1 -- 172.21.15.81:0/1400718160 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9d14062630 con 0x7f9d000a3270 2024-09-15T23:56:47.573 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 10 on host 'smithi081' 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 msgr2=0x7f9d000a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 0x7f9d000a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d14199380 tx=0x7f9cfc00b390 comp rx=0 tx=0).stop 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 msgr2=0x7f9d14197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d040314e0 tx=0x7f9d04031a30 comp rx=0 tx=0).stop 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 shutdown_connections 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9d000a3270 0x7f9d000a5730 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d141037c0 0x7f9d1419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:47.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d14102e00 0x7f9d141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:47.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 --2- 172.21.15.81:0/1400718160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d14101c00 0x7f9d14197e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:47.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 >> 172.21.15.81:0/1400718160 conn(0x7f9d140fd390 msgr2=0x7f9d14106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:56:47.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 shutdown_connections 2024-09-15T23:56:47.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-15T23:56:47.573+0000 7f9d19a15640 1 -- 172.21.15.81:0/1400718160 wait complete. 2024-09-15T23:56:48.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:47 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.245 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.246 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:47 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:47 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:48.272 DEBUG:teuthology.orchestra.run.smithi081:osd.10> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.10.service 2024-09-15T23:56:48.274 INFO:tasks.cephadm:Deploying osd.11 on smithi102 with /dev/vg_nvme/lv_1... 2024-09-15T23:56:48.274 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T23:56:48.525 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:56:48.924 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:48 smithi102 ceph-mon[34458]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:49.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:48 smithi081 ceph-mon[35060]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:48 smithi031 ceph-mon[31203]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:50.083 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 15 23:56:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-15T23:56:49.936+0000 7ff8cd067640 -1 osd.10 0 log_to_monitors true 2024-09-15T23:56:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:50 smithi102 ceph-mon[34458]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:50 smithi102 ceph-mon[34458]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:50 smithi102 ceph-mon[34458]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:50 smithi031 ceph-mon[31203]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:50 smithi031 ceph-mon[31203]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:50 smithi031 ceph-mon[31203]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.271 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:50 smithi081 ceph-mon[35060]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:51.271 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:50 smithi081 ceph-mon[35060]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.271 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:50 smithi081 ceph-mon[35060]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T23:56:51.341 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-15T23:56:51.367 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-09-15T23:56:51.617 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:52.018 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:51 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- 172.21.15.102:0/2666781521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67501037e0 msgr2=0x7f675010a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:52.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 --2- 172.21.15.102:0/2666781521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67501037e0 0x7f675010a070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6748009ca0 tx=0x7f674802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- 172.21.15.102:0/2666781521 shutdown_connections 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 --2- 172.21.15.102:0/2666781521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67501037e0 0x7f675010a070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 --2- 172.21.15.102:0/2666781521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6750102e20 0x7f67501032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 --2- 172.21.15.102:0/2666781521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750101c20 0x7f6750102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- 172.21.15.102:0/2666781521 >> 172.21.15.102:0/2666781521 conn(0x7f67500fd3b0 msgr2=0x7f67500ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- 172.21.15.102:0/2666781521 shutdown_connections 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- 172.21.15.102:0/2666781521 wait complete. 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 Processor -- start 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.100+0000 7f6755ea7640 1 -- start start 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750102e20 0x7f67500745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67501037e0 0x7f6750074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f675010d7d0 con 0x7f67501037e0 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f675010d650 con 0x7f6750102e20 2024-09-15T23:56:52.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f675010d950 con 0x7f6750101c20 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37748/0 (socket says 172.21.15.102:37748) 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 -- 172.21.15.102:0/1761991560 learned_addr learned my addr 172.21.15.102:0/1761991560 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6747fff640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750102e20 0x7f67500745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 -- 172.21.15.102:0/1761991560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750102e20 msgr2=0x7f67500745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750102e20 0x7f67500745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 -- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67501037e0 msgr2=0x7f6750074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67501037e0 0x7f6750074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 -- 172.21.15.102:0/1761991560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67500753e0 con 0x7f6750101c20 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674f7fe640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f674000c970 tx=0x7f674000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740004220 con 0x7f6750101c20 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67400043c0 con 0x7f6750101c20 2024-09-15T23:56:52.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67501a1fe0 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400047c0 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.101+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a23d0 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.102+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6750108ff0 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.104+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f67400079b0 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.104+0000 7f674d7fa640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 0x7f673c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.105+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f67400cef30 con 0x7f6750101c20 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.105+0000 7f6747fff640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 0x7f673c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.105+0000 7f6747fff640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 0x7f673c0a5700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6750075d10 tx=0x7f6738009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:56:52.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.109+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6740014030 con 0x7f6750101c20 2024-09-15T23:56:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T23:56:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T23:56:52.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:52.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:51 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:56:52.283+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6750062630 con 0x7f673c0a3240 2024-09-15T23:56:52.306 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-15T23:56:51.848+0000 7ff8c8fea640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T23:56:52.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-15T23:56:51.862+0000 7ff8c4e12640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-09-15T23:56:52.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:56:52.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:51 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:56:52.945 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi081 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:52.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:53.208 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:53.208 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi081 2024-09-15T23:56:53.208 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-15T23:56:53.208 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:56:53.208 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T23:56:53.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:53.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:53.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:53.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:53.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi081 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 4272M 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T23:56:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T23:56:53.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:56:54.043 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:56:54.043 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:56:54.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:54.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] boot 2024-09-15T23:56:54.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T23:56:54.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:54.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:53 smithi102 ceph-mon[34458]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T23:56:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:56:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:56:54.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] boot 2024-09-15T23:56:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T23:56:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:54.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:53 smithi031 ceph-mon[31203]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T23:56:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:56:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:56:54.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:56:54.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: osd.10 [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] boot 2024-09-15T23:56:54.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T23:56:54.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T23:56:54.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:53 smithi081 ceph-mon[35060]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T23:56:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/1259952489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]': finished 2024-09-15T23:56:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T23:56:55.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:54 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:55.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/1259952489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]': finished 2024-09-15T23:56:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T23:56:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:54 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:55.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/1259952489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]: dispatch 2024-09-15T23:56:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eead23d-bbb5-4fd5-adc7-3330798d55f4"}]': finished 2024-09-15T23:56:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T23:56:55.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:54 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:56.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:55 smithi102 ceph-mon[34458]: from='client.? 172.21.15.102:0/2290516904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:56.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:55 smithi031 ceph-mon[31203]: from='client.? 172.21.15.102:0/2290516904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:56.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:55 smithi081 ceph-mon[35060]: from='client.? 172.21.15.102:0/2290516904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T23:56:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:56 smithi031 ceph-mon[31203]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:56 smithi031 ceph-mon[31203]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T23:56:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:57.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:56 smithi081 ceph-mon[35060]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:56 smithi081 ceph-mon[35060]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T23:56:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:57.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:56 smithi102 ceph-mon[34458]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:56:57.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:56 smithi102 ceph-mon[34458]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T23:56:57.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:58.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:57 smithi031 ceph-mon[31203]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T23:56:58.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:57 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:58.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:57 smithi081 ceph-mon[35060]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T23:56:58.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:57 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:58.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:57 smithi102 ceph-mon[34458]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T23:56:58.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:57 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:56:59.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:56:58 smithi031 ceph-mon[31203]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-15T23:56:59.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:56:58 smithi081 ceph-mon[35060]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-15T23:56:59.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:56:58 smithi102 ceph-mon[34458]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-15T23:57:01.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:00 smithi102 ceph-mon[34458]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-15T23:57:01.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:00 smithi031 ceph-mon[31203]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-15T23:57:01.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:00 smithi081 ceph-mon[35060]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-15T23:57:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:02 smithi031 ceph-mon[31203]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T23:57:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:02 smithi081 ceph-mon[35060]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T23:57:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:02 smithi102 ceph-mon[34458]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T23:57:04.238 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:03 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T23:57:04.238 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:03 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:03 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T23:57:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:03 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:04.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:03 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T23:57:04.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:03 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:04 smithi102 ceph-mon[34458]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:57:05.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:04 smithi102 ceph-mon[34458]: Deploying daemon osd.11 on smithi102 2024-09-15T23:57:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:04 smithi031 ceph-mon[31203]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:57:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:04 smithi031 ceph-mon[31203]: Deploying daemon osd.11 on smithi102 2024-09-15T23:57:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:04 smithi081 ceph-mon[35060]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T23:57:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:04 smithi081 ceph-mon[35060]: Deploying daemon osd.11 on smithi102 2024-09-15T23:57:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:06 smithi031 ceph-mon[31203]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:06 smithi081 ceph-mon[35060]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T23:57:07.318 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:06 smithi102 ceph-mon[34458]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T23:57:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:08 smithi102 ceph-mon[34458]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:09.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:08 smithi031 ceph-mon[31203]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:08 smithi081 ceph-mon[35060]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:10.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:10.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:10.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:10.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:10.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:10.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:10.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:10.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:10.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:11.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:10 smithi031 ceph-mon[31203]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:10 smithi081 ceph-mon[35060]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:10 smithi102 ceph-mon[34458]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:12 smithi102 ceph-mon[34458]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:12 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:12 smithi031 ceph-mon[31203]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:12 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:13.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:12 smithi081 ceph-mon[35060]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:13.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:12 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:13.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.666+0000 7f674d7fa640 1 -- 172.21.15.102:0/1761991560 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6750062630 con 0x7f673c0a3240 2024-09-15T23:57:13.668 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 11 on host 'smithi102' 2024-09-15T23:57:13.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 msgr2=0x7f673c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:13.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 0x7f673c0a5700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6750075d10 tx=0x7f6738009110 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 msgr2=0x7f6750075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f674000c970 tx=0x7f674000cd70 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 shutdown_connections 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f673c0a3240 0x7f673c0a5700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67501037e0 0x7f6750074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6750102e20 0x7f67500745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 --2- 172.21.15.102:0/1761991560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6750101c20 0x7f6750075f90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.668+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 >> 172.21.15.102:0/1761991560 conn(0x7f67500fd3b0 msgr2=0x7f67501061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.669+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 shutdown_connections 2024-09-15T23:57:13.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:13.669+0000 7f6755ea7640 1 -- 172.21.15.102:0/1761991560 wait complete. 2024-09-15T23:57:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:13 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:13 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.330 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.331 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:13 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:14.360 DEBUG:teuthology.orchestra.run.smithi102:osd.11> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.11.service 2024-09-15T23:57:14.362 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-15T23:57:14.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T23:57:14.621 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:15.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:14 smithi031 ceph-mon[31203]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- 172.21.15.31:0/501224636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14102660 msgr2=0x7f5c14104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/501224636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14102660 0x7f5c14104a50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009900 tx=0x7f5c0802a410 comp rx=0 tx=0).stop 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- 172.21.15.31:0/501224636 shutdown_connections 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/501224636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c141078c0 0x7f5c14109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/501224636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c14104f90 0x7f5c14107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/501224636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14102660 0x7f5c14104a50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- 172.21.15.31:0/501224636 >> 172.21.15.31:0/501224636 conn(0x7f5c140fc280 msgr2=0x7f5c140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- 172.21.15.31:0/501224636 shutdown_connections 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- 172.21.15.31:0/501224636 wait complete. 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 Processor -- start 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.094+0000 7f5c1ad51640 1 -- start start 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c14102660 0x7f5c14101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c141078c0 0x7f5c14100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1410d790 con 0x7f5c14104f90 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1410d610 con 0x7f5c14102660 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c1ad51640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1410d910 con 0x7f5c141078c0 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49060/0 (socket says 172.21.15.31:49060) 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 -- 172.21.15.31:0/3862983015 learned_addr learned my addr 172.21.15.31:0/3862983015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c18aed640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c14102660 0x7f5c14101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c192ee640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c141078c0 0x7f5c14100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 -- 172.21.15.31:0/3862983015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c141078c0 msgr2=0x7f5c14100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c141078c0 0x7f5c14100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 -- 172.21.15.31:0/3862983015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c14102660 msgr2=0x7f5c14101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c14102660 0x7f5c14101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 -- 172.21.15.31:0/3862983015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c14101410 con 0x7f5c14104f90 2024-09-15T23:57:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c13fff640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9f0 tx=0x7f5c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.095+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000ccc0 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.096+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c000054d0 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.096+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c141a1fe0 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.096+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00010430 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.096+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141a2490 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.097+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c140fdd70 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.099+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5c00005000 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.099+0000 7f5c11ffb640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 0x7f5c040a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.100+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f5c000cf820 con 0x7f5c14104f90 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.100+0000 7f5c18aed640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 0x7f5c040a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.100+0000 7f5c18aed640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 0x7f5c040a5780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5c080314e0 tx=0x7f5c08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.103+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c000541b0 con 0x7f5c14104f90 2024-09-15T23:57:15.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.280+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5c1402da10 con 0x7f5c14104f90 2024-09-15T23:57:15.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.281+0000 7f5c11ffb640 1 -- 172.21.15.31:0/3862983015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f5c000052f0 con 0x7f5c14104f90 2024-09-15T23:57:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.282+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 msgr2=0x7f5c040a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.282+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 0x7f5c040a5780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5c080314e0 tx=0x7f5c08005a20 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.282+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 msgr2=0x7f5c14100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.282+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9f0 tx=0x7f5c0000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 shutdown_connections 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5c040a32c0 0x7f5c040a5780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c141078c0 0x7f5c14100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c14104f90 0x7f5c14100610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 --2- 172.21.15.31:0/3862983015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c14102660 0x7f5c14101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 >> 172.21.15.31:0/3862983015 conn(0x7f5c140fc280 msgr2=0x7f5c14102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 shutdown_connections 2024-09-15T23:57:15.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:15.283+0000 7f5c1ad51640 1 -- 172.21.15.31:0/3862983015 wait complete. 2024-09-15T23:57:15.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:14 smithi081 ceph-mon[35060]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:14 smithi102 ceph-mon[34458]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:16.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:15 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3862983015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:16.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 15 23:57:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-15T23:57:15.973+0000 7fbd5b4dd640 -1 osd.11 0 log_to_monitors true 2024-09-15T23:57:16.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:15 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3862983015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:16.226 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726444612,"num_in_osds":12,"osd_in_since":1726444614,"num_remapped_pgs":0} 2024-09-15T23:57:16.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:15 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3862983015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:17.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:16 smithi102 ceph-mon[34458]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:17.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:16 smithi102 ceph-mon[34458]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:16 smithi102 ceph-mon[34458]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T23:57:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:16 smithi031 ceph-mon[31203]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:16 smithi031 ceph-mon[31203]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:16 smithi031 ceph-mon[31203]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:16 smithi081 ceph-mon[35060]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:17.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:16 smithi081 ceph-mon[35060]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:16 smithi081 ceph-mon[35060]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T23:57:17.497 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.017+0000 7f0d48591640 1 -- 172.21.15.31:0/77226508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 msgr2=0x7f0d40103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.017+0000 7f0d48591640 1 --2- 172.21.15.31:0/77226508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40103280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d3002a410 comp rx=0 tx=0).stop 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 -- 172.21.15.31:0/77226508 shutdown_connections 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 --2- 172.21.15.31:0/77226508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d401037c0 0x7f0d4010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 --2- 172.21.15.31:0/77226508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40103280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 --2- 172.21.15.31:0/77226508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d40101c00 0x7f0d40102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 -- 172.21.15.31:0/77226508 >> 172.21.15.31:0/77226508 conn(0x7f0d400fd390 msgr2=0x7f0d400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:18.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 -- 172.21.15.31:0/77226508 shutdown_connections 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.018+0000 7f0d48591640 1 -- 172.21.15.31:0/77226508 wait complete. 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 Processor -- start 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 -- start start 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d40101c00 0x7f0d40197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d401037c0 0x7f0d4019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d760 con 0x7f0d40102e00 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d5e0 con 0x7f0d40101c00 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d48591640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d8e0 con 0x7f0d401037c0 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49070/0 (socket says 172.21.15.31:49070) 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 -- 172.21.15.31:0/1891638461 learned_addr learned my addr 172.21.15.31:0/1891638461 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:18.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d46b2e640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d401037c0 0x7f0d4019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d4632d640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d40101c00 0x7f0d40197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 -- 172.21.15.31:0/1891638461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d401037c0 msgr2=0x7f0d4019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d401037c0 0x7f0d4019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 -- 172.21.15.31:0/1891638461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d40101c00 msgr2=0x7f0d40197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d40101c00 0x7f0d40197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.019+0000 7f0d45b2c640 1 -- 172.21.15.31:0/1891638461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4019fa20 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d45b2c640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d300314e0 tx=0x7f0d30003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3003f070 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4019fcb0 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d30031c40 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30035030 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.020+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401a0190 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.021+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d40108ff0 con 0x7f0d40102e00 2024-09-15T23:57:18.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.022+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0d30042070 con 0x7f0d40102e00 2024-09-15T23:57:18.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.023+0000 7f0d377fe640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 0x7f0d2c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:18.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.023+0000 7f0d4632d640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 0x7f0d2c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:18.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.023+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f0d300efb60 con 0x7f0d40102e00 2024-09-15T23:57:18.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.023+0000 7f0d4632d640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 0x7f0d2c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d40102c60 tx=0x7f0d3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:18.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.027+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d300bc750 con 0x7f0d40102e00 2024-09-15T23:57:18.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.201+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0d40062df0 con 0x7f0d40102e00 2024-09-15T23:57:18.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T23:57:18.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T23:57:18.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:18.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:18.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:17 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.202+0000 7f0d377fe640 1 -- 172.21.15.31:0/1891638461 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f0d300c0a50 con 0x7f0d40102e00 2024-09-15T23:57:18.205 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:18.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 msgr2=0x7f0d2c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:18.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 0x7f0d2c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d40102c60 tx=0x7f0d3c005f90 comp rx=0 tx=0).stop 2024-09-15T23:57:18.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 msgr2=0x7f0d40198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d300314e0 tx=0x7f0d30003c10 comp rx=0 tx=0).stop 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 shutdown_connections 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f0d2c0a3270 0x7f0d2c0a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d401037c0 0x7f0d4019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d40102e00 0x7f0d40198330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 --2- 172.21.15.31:0/1891638461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d40101c00 0x7f0d40197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 >> 172.21.15.31:0/1891638461 conn(0x7f0d400fd390 msgr2=0x7f0d401061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 shutdown_connections 2024-09-15T23:57:18.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:18.205+0000 7f0d48591640 1 -- 172.21.15.31:0/1891638461 wait complete. 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:17 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.370 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-15T23:57:17.961+0000 7fbd57460640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T23:57:18.371 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-15T23:57:17.974+0000 7fbd53288640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-15T23:57:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:18.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:17 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:19.116 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726444612,"num_in_osds":12,"osd_in_since":1726444614,"num_remapped_pgs":0} 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: Detected new or changed devices on smithi102 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:18 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1891638461' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: Detected new or changed devices on smithi102 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:18 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1891638461' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: Detected new or changed devices on smithi102 2024-09-15T23:57:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-15T23:57:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-15T23:57:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T23:57:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:18 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1891638461' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:20.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: purged_snaps scrub starts 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: purged_snaps scrub ok 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] boot 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T23:57:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:19 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: purged_snaps scrub starts 2024-09-15T23:57:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: purged_snaps scrub ok 2024-09-15T23:57:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] boot 2024-09-15T23:57:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T23:57:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:19 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: purged_snaps scrub starts 2024-09-15T23:57:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: purged_snaps scrub ok 2024-09-15T23:57:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: osd.11 [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] boot 2024-09-15T23:57:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T23:57:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:19 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T23:57:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 -- 172.21.15.31:0/823863326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508107900 msgr2=0x7fc508109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 --2- 172.21.15.31:0/823863326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508107900 0x7fc508109d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc500009be0 tx=0x7fc50002a6c0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 -- 172.21.15.31:0/823863326 shutdown_connections 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 --2- 172.21.15.31:0/823863326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc508107900 0x7fc508109d50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 --2- 172.21.15.31:0/823863326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508104fd0 0x7fc5081073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 --2- 172.21.15.31:0/823863326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5081026a0 0x7fc508104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 -- 172.21.15.31:0/823863326 >> 172.21.15.31:0/823863326 conn(0x7fc5080fc2a0 msgr2=0x7fc5080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 -- 172.21.15.31:0/823863326 shutdown_connections 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.077+0000 7fc50eba6640 1 -- 172.21.15.31:0/823863326 wait complete. 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 Processor -- start 2024-09-15T23:57:21.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 -- start start 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508104fd0 0x7fc508198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 0x7fc50819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810d770 con 0x7fc5081026a0 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810d5f0 con 0x7fc508107900 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50eba6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810d8f0 con 0x7fc508104fd0 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49090/0 (socket says 172.21.15.31:49090) 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.078+0000 7fc50c942640 1 -- 172.21.15.31:0/3430958264 learned_addr learned my addr 172.21.15.31:0/3430958264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50d143640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 0x7fc50819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 -- 172.21.15.31:0/3430958264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508104fd0 msgr2=0x7fc508198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:21.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508104fd0 0x7fc508198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 -- 172.21.15.31:0/3430958264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 msgr2=0x7fc50819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 0x7fc50819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 -- 172.21.15.31:0/3430958264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50819fa60 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50d143640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 0x7fc50819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50c942640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e9c0 tx=0x7fc4fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00ccc0 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819fd50 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4fc0054d0 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc010430 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.079+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a0250 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.080+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5080fdd90 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.081+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc4fc0046c0 con 0x7fc5081026a0 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.082+0000 7fc505ffb640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 0x7fc4f80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:21.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.082+0000 7fc507fff640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 0x7fc4f80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:21.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.082+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc4fc0cf880 con 0x7fc5081026a0 2024-09-15T23:57:21.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.083+0000 7fc507fff640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 0x7fc4f80a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4005fd0 tx=0x7fc4f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:21.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.086+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4fc09c9a0 con 0x7fc5081026a0 2024-09-15T23:57:21.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:20 smithi031 ceph-mon[31203]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:21.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.261+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc508062f30 con 0x7fc5081026a0 2024-09-15T23:57:21.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.261+0000 7fc505ffb640 1 -- 172.21.15.31:0/3430958264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fc4fc09c2a0 con 0x7fc5081026a0 2024-09-15T23:57:21.262 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:21.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 msgr2=0x7fc4f80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 0x7fc4f80a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4005fd0 tx=0x7fc4f4005850 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 msgr2=0x7fc508197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e9c0 tx=0x7fc4fc00ee80 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 shutdown_connections 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc4f80a3270 0x7fc4f80a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc508107900 0x7fc50819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508104fd0 0x7fc508198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 --2- 172.21.15.31:0/3430958264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081026a0 0x7fc508197e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 >> 172.21.15.31:0/3430958264 conn(0x7fc5080fc2a0 msgr2=0x7fc508103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 shutdown_connections 2024-09-15T23:57:21.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:21.264+0000 7fc50eba6640 1 -- 172.21.15.31:0/3430958264 wait complete. 2024-09-15T23:57:21.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:20 smithi081 ceph-mon[35060]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:20 smithi102 ceph-mon[34458]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-15T23:57:22.103 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726444638,"num_in_osds":12,"osd_in_since":1726444614,"num_remapped_pgs":0} 2024-09-15T23:57:22.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:57:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:21 smithi031 ceph-mon[31203]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T23:57:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:21 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3430958264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:22.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:21 smithi081 ceph-mon[35060]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T23:57:22.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:21 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3430958264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:22.367 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:22.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:21 smithi102 ceph-mon[34458]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T23:57:22.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:21 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3430958264' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T23:57:22.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 -- 172.21.15.31:0/931422863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c102570 msgr2=0x7fe46c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 --2- 172.21.15.31:0/931422863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c102570 0x7fe46c1029b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002a480 comp rx=0 tx=0).stop 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 -- 172.21.15.31:0/931422863 shutdown_connections 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 --2- 172.21.15.31:0/931422863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 0x7fe46c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 --2- 172.21.15.31:0/931422863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c102570 0x7fe46c1029b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 --2- 172.21.15.31:0/931422863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c100e20 0x7fe46c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 -- 172.21.15.31:0/931422863 >> 172.21.15.31:0/931422863 conn(0x7fe46c0fc7d0 msgr2=0x7fe46c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 -- 172.21.15.31:0/931422863 shutdown_connections 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.863+0000 7fe4729c5640 1 -- 172.21.15.31:0/931422863 wait complete. 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 Processor -- start 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 -- start start 2024-09-15T23:57:22.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c102570 0x7fe46c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 0x7fe46c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c110e50 con 0x7fe46c100e20 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c110cd0 con 0x7fe46c102ef0 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe4729c5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c110fd0 con 0x7fe46c102570 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39630/0 (socket says 172.21.15.31:39630) 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.864+0000 7fe46bfff640 1 -- 172.21.15.31:0/446028141 learned_addr learned my addr 172.21.15.31:0/446028141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46b7fe640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c102570 0x7fe46c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:22.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe470f62640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 0x7fe46c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 -- 172.21.15.31:0/446028141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c102570 msgr2=0x7fe46c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c102570 0x7fe46c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 -- 172.21.15.31:0/446028141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 msgr2=0x7fe46c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 0x7fe46c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 -- 172.21.15.31:0/446028141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c1a6980 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe46bfff640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b4a0 tx=0x7fe45c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c014070 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45c010ac0 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c017c60 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.865+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a6c70 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.866+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a71b0 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.867+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c101200 con 0x7fe46c100e20 2024-09-15T23:57:22.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.868+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe45c017070 con 0x7fe46c100e20 2024-09-15T23:57:22.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.868+0000 7fe4697fa640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 0x7fe4540a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:22.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.869+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe45c0ce7c0 con 0x7fe46c100e20 2024-09-15T23:57:22.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.869+0000 7fe46b7fe640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 0x7fe4540a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:22.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.869+0000 7fe46b7fe640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 0x7fe4540a5850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0fe710 tx=0x7fe4600095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:22.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:22.873+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe45c09b980 con 0x7fe46c100e20 2024-09-15T23:57:23.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.050+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe46c10a0a0 con 0x7fe46c100e20 2024-09-15T23:57:23.051 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:22 smithi031 ceph-mon[31203]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:23.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.051+0000 7fe4697fa640 1 -- 172.21.15.31:0/446028141 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18406 (secure 0 0 0) 0x7fe45c09b280 con 0x7fe46c100e20 2024-09-15T23:57:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:23.052 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:57:20.968163+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 msgr2=0x7fe4540a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 0x7fe4540a5850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0fe710 tx=0x7fe4600095f0 comp rx=0 tx=0).stop 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 msgr2=0x7fe46c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b4a0 tx=0x7fe45c00b960 comp rx=0 tx=0).stop 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 shutdown_connections 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4540a3390 0x7fe4540a5850 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:23.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c102ef0 0x7fe46c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe46c102570 0x7fe46c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 --2- 172.21.15.31:0/446028141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe46c100e20 0x7fe46c0fcc80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 >> 172.21.15.31:0/446028141 conn(0x7fe46c0fc7d0 msgr2=0x7fe46c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.053+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 shutdown_connections 2024-09-15T23:57:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:23.054+0000 7fe4729c5640 1 -- 172.21.15.31:0/446028141 wait complete. 2024-09-15T23:57:23.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:22 smithi081 ceph-mon[35060]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:22 smithi102 ceph-mon[34458]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:23.787 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T23:53:27.616831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T23:57:23.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-15T23:57:24.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:23 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/446028141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:57:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:24.272 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:23 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/446028141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:57:24.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:23 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/446028141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:57:24.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 -- 172.21.15.31:0/932852402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 msgr2=0x7f9c741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/932852402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c741029b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c6402a480 comp rx=0 tx=0).stop 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 -- 172.21.15.31:0/932852402 shutdown_connections 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/932852402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 0x7f9c7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/932852402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c741029b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/932852402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 0x7f9c74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 -- 172.21.15.31:0/932852402 >> 172.21.15.31:0/932852402 conn(0x7f9c740fc7d0 msgr2=0x7f9c740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 -- 172.21.15.31:0/932852402 shutdown_connections 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.524+0000 7f9c7b68e640 1 -- 172.21.15.31:0/932852402 wait complete. 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 Processor -- start 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 -- start start 2024-09-15T23:57:24.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 0x7f9c740fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 0x7f9c740fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74110e00 con 0x7f9c74102570 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74110c80 con 0x7f9c74102ef0 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7b68e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74110f80 con 0x7f9c74100e20 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39644/0 (socket says 172.21.15.31:39644) 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 -- 172.21.15.31:0/2711004864 learned_addr learned my addr 172.21.15.31:0/2711004864 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c7942a640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 0x7f9c740fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c79c2b640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 0x7f9c740fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 -- 172.21.15.31:0/2711004864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 msgr2=0x7f9c740fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:24.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 0x7f9c740fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 -- 172.21.15.31:0/2711004864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 msgr2=0x7f9c740fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 0x7f9c740fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.525+0000 7f9c78c29640 1 -- 172.21.15.31:0/2711004864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c741a6970 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c78c29640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6403f030 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c741a6c00 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c64003cb0 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64035070 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.526+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c741a70e0 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.527+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c74101200 con 0x7f9c74102570 2024-09-15T23:57:24.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.528+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9c64042030 con 0x7f9c74102570 2024-09-15T23:57:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.529+0000 7f9c6a7fc640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 0x7f9c600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.529+0000 7f9c7942a640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 0x7f9c600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.529+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9c640efc50 con 0x7f9c74102570 2024-09-15T23:57:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.529+0000 7f9c7942a640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 0x7f9c600a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000bbe0 tx=0x7f9c7000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.532+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c640bcd70 con 0x7f9c74102570 2024-09-15T23:57:24.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.708+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c7410a0a0 con 0x7f9c74102570 2024-09-15T23:57:24.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.708+0000 7f9c6a7fc640 1 -- 172.21.15.31:0/2711004864 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f9c640bc670 con 0x7f9c74102570 2024-09-15T23:57:24.709 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 msgr2=0x7f9c600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 0x7f9c600a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000bbe0 tx=0x7f9c7000b390 comp rx=0 tx=0).stop 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 msgr2=0x7f9c740fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c640042f0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 shutdown_connections 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9c600a3240 0x7f9c600a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c74102ef0 0x7f9c740fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c74102570 0x7f9c740fd1c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 --2- 172.21.15.31:0/2711004864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c74100e20 0x7f9c740fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 >> 172.21.15.31:0/2711004864 conn(0x7f9c740fc7d0 msgr2=0x7f9c7406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.710+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 shutdown_connections 2024-09-15T23:57:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:24.711+0000 7f9c7b68e640 1 -- 172.21.15.31:0/2711004864 wait complete. 2024-09-15T23:57:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:24 smithi031 ceph-mon[31203]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:24 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2711004864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:57:25.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:24 smithi081 ceph-mon[35060]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:25.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:24 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2711004864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:57:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:24 smithi102 ceph-mon[34458]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:24 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2711004864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.b on smithi081 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.e on smithi081 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.h on smithi081 2024-09-15T23:57:25.438 INFO:tasks.cephadm:Adding mds.k on smithi081 2024-09-15T23:57:25.439 INFO:tasks.cephadm:Adding mds.c on smithi102 2024-09-15T23:57:25.439 INFO:tasks.cephadm:Adding mds.f on smithi102 2024-09-15T23:57:25.439 INFO:tasks.cephadm:Adding mds.i on smithi102 2024-09-15T23:57:25.439 INFO:tasks.cephadm:Adding mds.l on smithi102 2024-09-15T23:57:25.439 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l' 2024-09-15T23:57:25.688 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2208109508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 msgr2=0x7f6bc4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2208109508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4103280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009900 tx=0x7f6bb802a410 comp rx=0 tx=0).stop 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2208109508 shutdown_connections 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2208109508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc41037c0 0x7f6bc410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2208109508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4103280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2208109508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 0x7f6bc4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2208109508 >> 172.21.15.102:0/2208109508 conn(0x7f6bc40fd390 msgr2=0x7f6bc40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2208109508 shutdown_connections 2024-09-15T23:57:26.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.176+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2208109508 wait complete. 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 Processor -- start 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 -- start start 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 0x7f6bc4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc41037c0 0x7f6bc419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc410d730 con 0x7f6bc4101c00 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc410d5b0 con 0x7f6bc41037c0 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc8e16640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc410d8b0 con 0x7f6bc4102e00 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50848/0 (socket says 172.21.15.102:50848) 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.177+0000 7f6bc1d9b640 1 -- 172.21.15.102:0/2029412954 learned_addr learned my addr 172.21.15.102:0/2029412954 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc259c640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 0x7f6bc4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 -- 172.21.15.102:0/2029412954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc41037c0 msgr2=0x7f6bc419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:26.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc41037c0 0x7f6bc419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 -- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 msgr2=0x7f6bc4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 0x7f6bc4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 -- 172.21.15.102:0/2029412954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419fa70 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc1d9b640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8004770 tx=0x7f6bb80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb803f030 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bb8003cb0 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419fd00 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8035070 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.178+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a01d0 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.179+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc41090c0 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.180+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6bb8042030 con 0x7f6bc4102e00 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.181+0000 7f6bb37fe640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 0x7f6bac0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:26.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.181+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6bb80f00c0 con 0x7f6bc4102e00 2024-09-15T23:57:26.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.181+0000 7f6bc259c640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 0x7f6bac0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:26.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.182+0000 7f6bc259c640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 0x7f6bac0a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005fd0 tx=0x7f6bb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:26.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.184+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bb80bd280 con 0x7f6bc4102e00 2024-09-15T23:57:26.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.360+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}) -- 0x7f6bc40627f0 con 0x7f6bac0a32f0 2024-09-15T23:57:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.367+0000 7f6bb37fe640 1 -- 172.21.15.102:0/2029412954 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6bc40627f0 con 0x7f6bac0a32f0 2024-09-15T23:57:26.369 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mds.all update... 2024-09-15T23:57:26.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 msgr2=0x7f6bac0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:26.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 0x7f6bac0a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005fd0 tx=0x7f6bb4005850 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 msgr2=0x7f6bc4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8004770 tx=0x7f6bb80042f0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 shutdown_connections 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6bac0a32f0 0x7f6bac0a57b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc41037c0 0x7f6bc419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4102e00 0x7f6bc4198380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 --2- 172.21.15.102:0/2029412954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc4101c00 0x7f6bc4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 >> 172.21.15.102:0/2029412954 conn(0x7f6bc40fd390 msgr2=0x7f6bc4106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 shutdown_connections 2024-09-15T23:57:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-15T23:57:26.370+0000 7f6bc8e16640 1 -- 172.21.15.102:0/2029412954 wait complete. 2024-09-15T23:57:27.040 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.a.service 2024-09-15T23:57:27.043 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.d.service 2024-09-15T23:57:27.085 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.g.service 2024-09-15T23:57:27.128 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.j.service 2024-09-15T23:57:27.172 DEBUG:teuthology.orchestra.run.smithi081:mds.b> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.b.service 2024-09-15T23:57:27.175 DEBUG:teuthology.orchestra.run.smithi081:mds.e> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.e.service 2024-09-15T23:57:27.218 DEBUG:teuthology.orchestra.run.smithi081:mds.h> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.h.service 2024-09-15T23:57:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:26 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.261 DEBUG:teuthology.orchestra.run.smithi081:mds.k> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.k.service 2024-09-15T23:57:27.279 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:27.279 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:27.280 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:26 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.281 DEBUG:teuthology.orchestra.run.smithi102:mds.c> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.c.service 2024-09-15T23:57:27.283 DEBUG:teuthology.orchestra.run.smithi102:mds.f> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.f.service 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:27.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:27.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:26 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:27.304 DEBUG:teuthology.orchestra.run.smithi102:mds.i> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.i.service 2024-09-15T23:57:27.347 DEBUG:teuthology.orchestra.run.smithi102:mds.l> sudo journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.l.service 2024-09-15T23:57:27.390 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-15T23:57:27.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:28.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:28 smithi031 ceph-mon[31203]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:57:28.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:28 smithi031 ceph-mon[31203]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-15T23:57:28.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:28 smithi031 ceph-mon[31203]: Deploying daemon mds.l on smithi102 2024-09-15T23:57:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 -- 172.21.15.31:0/3137160276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604101420 msgr2=0x7fa60410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 --2- 172.21.15.31:0/3137160276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604101420 0x7fa60410d950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009cd0 tx=0x7fa5fc02a8e0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 -- 172.21.15.31:0/3137160276 shutdown_connections 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 --2- 172.21.15.31:0/3137160276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604101420 0x7fa60410d950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 --2- 172.21.15.31:0/3137160276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604100a80 0x7fa604100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 --2- 172.21.15.31:0/3137160276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604106a80 0x7fa604106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 -- 172.21.15.31:0/3137160276 >> 172.21.15.31:0/3137160276 conn(0x7fa6040fc7a0 msgr2=0x7fa6040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 -- 172.21.15.31:0/3137160276 shutdown_connections 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 -- 172.21.15.31:0/3137160276 wait complete. 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.124+0000 7fa60c35d640 1 Processor -- start 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 -- start start 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604101420 0x7fa604198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604106a80 0x7fa60419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:28.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111e50 con 0x7fa604100a80 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111cd0 con 0x7fa604101420 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60c35d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111fd0 con 0x7fa604106a80 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39664/0 (socket says 172.21.15.31:39664) 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 -- 172.21.15.31:0/1258045357 learned_addr learned my addr 172.21.15.31:0/1258045357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa6098f8640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604101420 0x7fa604198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a8fa640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604106a80 0x7fa60419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 -- 172.21.15.31:0/1258045357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604106a80 msgr2=0x7fa60419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604106a80 0x7fa60419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 -- 172.21.15.31:0/1258045357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604101420 msgr2=0x7fa604198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604101420 0x7fa604198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 -- 172.21.15.31:0/1258045357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60419d2e0 con 0x7fa604100a80 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.125+0000 7fa60a0f9640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d950 tx=0x7fa60000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.126+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600017070 con 0x7fa604100a80 2024-09-15T23:57:28.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.126+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa600009c30 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.126+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60419d5d0 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.126+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60000f620 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.126+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041a4eb0 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.127+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa604100ee0 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.128+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa6000109a0 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.128+0000 7fa5fb7fe640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 0x7fa5f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.128+0000 7fa6098f8640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 0x7fa5f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.129+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa6000cf050 con 0x7fa604100a80 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.129+0000 7fa6098f8640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 0x7fa5f00a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6040feda0 tx=0x7fa5f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:28.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.133+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa600009de0 con 0x7fa604100a80 2024-09-15T23:57:28.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:28 smithi081 ceph-mon[35060]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:57:28.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:28 smithi081 ceph-mon[35060]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-15T23:57:28.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:28 smithi081 ceph-mon[35060]: Deploying daemon mds.l on smithi102 2024-09-15T23:57:28.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:28 smithi102 ceph-mon[34458]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:57:28.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:28 smithi102 ceph-mon[34458]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-15T23:57:28.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:28 smithi102 ceph-mon[34458]: Deploying daemon mds.l on smithi102 2024-09-15T23:57:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.343+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa604109010 con 0x7fa604100a80 2024-09-15T23:57:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.343+0000 7fa5fb7fe640 1 -- 172.21.15.31:0/1258045357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fa60009c210 con 0x7fa604100a80 2024-09-15T23:57:28.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:28.345 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"btime":"2024-09-15T23:50:23:938450+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-15T23:57:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 msgr2=0x7fa5f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 0x7fa5f00a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6040feda0 tx=0x7fa5f400b390 comp rx=0 tx=0).stop 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 msgr2=0x7fa604198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d950 tx=0x7fa60000de10 comp rx=0 tx=0).stop 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 shutdown_connections 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa5f00a3240 0x7fa5f00a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa604106a80 0x7fa60419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa604101420 0x7fa604198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 --2- 172.21.15.31:0/1258045357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa604100a80 0x7fa604198330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 >> 172.21.15.31:0/1258045357 conn(0x7fa6040fc7a0 msgr2=0x7fa6040fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 shutdown_connections 2024-09-15T23:57:28.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:28.346+0000 7fa60c35d640 1 -- 172.21.15.31:0/1258045357 wait complete. 2024-09-15T23:57:29.003 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:28 smithi102 podman[63846]: 2024-09-15 23:57:28.918664421 +0000 UTC m=+0.141048810 container create 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-15T23:57:29.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:29 smithi031 ceph-mon[31203]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:29.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:29 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1258045357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:29.263 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:29 smithi102 ceph-mon[34458]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:29.264 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:29 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1258045357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:29.264 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 podman[63846]: 2024-09-15 23:57:29.171898474 +0000 UTC m=+0.394282873 container init 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T23:57:29.264 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 podman[63846]: 2024-09-15 23:57:29.174769223 +0000 UTC m=+0.397153616 container start 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-15T23:57:29.264 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 ceph-mds[63865]: -- 172.21.15.102:0/3889700134 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9839a3860 con 0x55a983981000 2024-09-15T23:57:29.264 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: starting mds.l at 2024-09-15T23:57:29.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:29 smithi081 ceph-mon[35060]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:29.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:29 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1258045357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:29.620 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 bash[63846]: 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d 2024-09-15T23:57:29.620 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 15 23:57:29 smithi102 systemd[1]: Started Ceph mds.l for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6832/4135774449,v1:172.21.15.102:6833/4135774449] up:boot 2024-09-15T23:57:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: fsmap 1 up:standby 2024-09-15T23:57:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T23:57:30.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:30.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:30 smithi031 ceph-mon[31203]: Deploying daemon mds.k on smithi081 2024-09-15T23:57:30.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6832/4135774449,v1:172.21.15.102:6833/4135774449] up:boot 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: fsmap 1 up:standby 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:30.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:30 smithi081 ceph-mon[35060]: Deploying daemon mds.k on smithi081 2024-09-15T23:57:30.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:30.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6832/4135774449,v1:172.21.15.102:6833/4135774449] up:boot 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: fsmap 1 up:standby 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:30.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:30 smithi102 ceph-mon[34458]: Deploying daemon mds.k on smithi081 2024-09-15T23:57:31.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:31.432 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:31.557 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 systemd[1]: Starting Ceph mds.k for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:31.844 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 podman[63797]: 2024-09-15 23:57:31.478450162 +0000 UTC m=+0.015832433 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:31.845 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 podman[63797]: 2024-09-15 23:57:31.602332759 +0000 UTC m=+0.139715012 container create ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T23:57:31.845 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 podman[63797]: 2024-09-15 23:57:31.84430867 +0000 UTC m=+0.381690932 container init ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 -- 172.21.15.31:0/1103524514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 msgr2=0x7f8fa01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1103524514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01067e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009900 tx=0x7f8f8802a410 comp rx=0 tx=0).stop 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 -- 172.21.15.31:0/1103524514 shutdown_connections 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1103524514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0106f10 0x7f8fa010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1103524514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01067e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1103524514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0101de0 0x7f8fa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 -- 172.21.15.31:0/1103524514 >> 172.21.15.31:0/1103524514 conn(0x7f8fa00fd8c0 msgr2=0x7f8fa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 -- 172.21.15.31:0/1103524514 shutdown_connections 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.915+0000 7f8fa4980640 1 -- 172.21.15.31:0/1103524514 wait complete. 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 Processor -- start 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 -- start start 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 0x7f8fa01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0106f10 0x7f8fa019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0111e20 con 0x7f8fa0102790 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0111ca0 con 0x7f8fa0106f10 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8fa4980640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0111fa0 con 0x7f8fa0101de0 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36756/0 (socket says 172.21.15.31:36756) 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 -- 172.21.15.31:0/1325013790 learned_addr learned my addr 172.21.15.31:0/1325013790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9e59c640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 0x7f8fa01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9ed9d640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0106f10 0x7f8fa019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 -- 172.21.15.31:0/1325013790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 msgr2=0x7f8fa01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 0x7f8fa01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 -- 172.21.15.31:0/1325013790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0106f10 msgr2=0x7f8fa019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0106f10 0x7f8fa019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9dd9b640 1 -- 172.21.15.31:0/1325013790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa019d350 con 0x7f8fa0102790 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.916+0000 7f8f9e59c640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 0x7f8fa01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8f9dd9b640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f88005940 tx=0x7f8f88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8803f030 con 0x7f8fa0102790 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f88003aa0 con 0x7f8fa0102790 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa019d5e0 con 0x7f8fa0102790 2024-09-15T23:57:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88035070 con 0x7f8fa0102790 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.917+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a4f20 con 0x7f8fa0102790 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.918+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa01032c0 con 0x7f8fa0102790 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.919+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8f88042030 con 0x7f8fa0102790 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.920+0000 7f8f977fe640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 0x7f8f8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.920+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f8f880efe80 con 0x7f8fa0102790 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.920+0000 7f8f9e59c640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 0x7f8f8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.921+0000 7f8f9e59c640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 0x7f8f8c0a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f900058c0 tx=0x7f8f90005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:31.923+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f880bd040 con 0x7f8fa0102790 2024-09-15T23:57:32.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.152+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa01021c0 con 0x7f8fa0102790 2024-09-15T23:57:32.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.153+0000 7f8f977fe640 1 -- 172.21.15.31:0/1325013790 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f8fa01021c0 con 0x7f8fa0102790 2024-09-15T23:57:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:32.154 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"btime":"2024-09-15T23:57:29:202676+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T23:57:32.154 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-09-15T23:57:32.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 msgr2=0x7f8f8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 0x7f8f8c0a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f900058c0 tx=0x7f8f90005850 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 msgr2=0x7f8fa01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f88005940 tx=0x7f8f88004270 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 shutdown_connections 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8f8c0a3240 0x7f8f8c0a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fa0106f10 0x7f8fa019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa0102790 0x7f8fa01988e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 --2- 172.21.15.31:0/1325013790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0101de0 0x7f8fa01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 >> 172.21.15.31:0/1325013790 conn(0x7f8fa00fd8c0 msgr2=0x7f8fa00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 shutdown_connections 2024-09-15T23:57:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:32.155+0000 7f8fa4980640 1 -- 172.21.15.31:0/1325013790 wait complete. 2024-09-15T23:57:32.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 podman[63797]: 2024-09-15 23:57:31.84729379 +0000 UTC m=+0.384676043 container start ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-15T23:57:32.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 ceph-mds[63816]: -- 172.21.15.81:0/377737699 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa67ea5c20 con 0x55aa67e82800 2024-09-15T23:57:32.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: starting mds.k at 2024-09-15T23:57:32.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 bash[63797]: ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b 2024-09-15T23:57:32.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 15 23:57:31 smithi081 systemd[1]: Started Ceph mds.k for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: Deploying daemon mds.j on smithi031 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1325013790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6832/958737504,v1:172.21.15.81:6833/958737504] up:boot 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: fsmap 2 up:standby 2024-09-15T23:57:33.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:33 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: Deploying daemon mds.j on smithi031 2024-09-15T23:57:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1325013790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:33.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6832/958737504,v1:172.21.15.81:6833/958737504] up:boot 2024-09-15T23:57:33.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: fsmap 2 up:standby 2024-09-15T23:57:33.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:33 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T23:57:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: Deploying daemon mds.j on smithi031 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1325013790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6832/958737504,v1:172.21.15.81:6833/958737504] up:boot 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: fsmap 2 up:standby 2024-09-15T23:57:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:33 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T23:57:34.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:34.962 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:34 smithi031 systemd[1]: Starting Ceph mds.j for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:35.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:35 smithi031 ceph-mon[31203]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:35.253 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:34 smithi031 podman[70983]: 2024-09-15 23:57:34.868260012 +0000 UTC m=+0.016436345 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:35.253 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:34 smithi031 podman[70983]: 2024-09-15 23:57:34.98332361 +0000 UTC m=+0.131499930 container create cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T23:57:35.254 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 podman[70983]: 2024-09-15 23:57:35.252997033 +0000 UTC m=+0.401173351 container init cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-15T23:57:35.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:35 smithi081 ceph-mon[35060]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:35 smithi102 ceph-mon[34458]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:35.438 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:35.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 podman[70983]: 2024-09-15 23:57:35.256451799 +0000 UTC m=+0.404628125 container start cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:57:35.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 ceph-mds[71032]: -- 172.21.15.31:0/2924551617 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561742025860 con 0x561742003400 2024-09-15T23:57:35.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j[71016]: starting mds.j at 2024-09-15T23:57:35.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 bash[70983]: cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 2024-09-15T23:57:35.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 15 23:57:35 smithi031 systemd[1]: Started Ceph mds.j for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 -- 172.21.15.31:0/1260577830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 msgr2=0x7f001c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 --2- 172.21.15.31:0/1260577830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c103120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f000c009900 tx=0x7f000c02a480 comp rx=0 tx=0).stop 2024-09-15T23:57:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 -- 172.21.15.31:0/1260577830 shutdown_connections 2024-09-15T23:57:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 --2- 172.21.15.31:0/1260577830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 --2- 172.21.15.31:0/1260577830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 --2- 172.21.15.31:0/1260577830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c103120 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 -- 172.21.15.31:0/1260577830 >> 172.21.15.31:0/1260577830 conn(0x7f001c0fc7a0 msgr2=0x7f001c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 -- 172.21.15.31:0/1260577830 shutdown_connections 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.991+0000 7f0021599640 1 -- 172.21.15.31:0/1260577830 wait complete. 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 Processor -- start 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 -- start start 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10fc60 con 0x7f001c102d40 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10fae0 con 0x7f001c10c170 2024-09-15T23:57:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.992+0000 7f0021599640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10fde0 con 0x7f001c103660 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36784/0 (socket says 172.21.15.31:36784) 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 -- 172.21.15.31:0/1231336570 learned_addr learned my addr 172.21.15.31:0/1231336570 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001a7fc640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001b7fe640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 -- 172.21.15.31:0/1231336570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 msgr2=0x7f001c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 -- 172.21.15.31:0/1231336570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 msgr2=0x7f001c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 -- 172.21.15.31:0/1231336570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c19b110 con 0x7f001c102d40 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001b7fe640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:57:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001a7fc640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f001affd640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f000c031440 tx=0x7f000c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c03f070 con 0x7f001c102d40 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f000c003e40 con 0x7f001c102d40 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.993+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c19b3a0 con 0x7f001c102d40 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.994+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c035030 con 0x7f001c102d40 2024-09-15T23:57:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.994+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c1a2ce0 con 0x7f001c102d40 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.994+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001c103120 con 0x7f001c102d40 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.996+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f000c042070 con 0x7f001c102d40 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.996+0000 7f0003fff640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 0x7f00040a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.997+0000 7f001a7fc640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 0x7f00040a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.997+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f000c0f0370 con 0x7f001c102d40 2024-09-15T23:57:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:35.997+0000 7f001a7fc640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 0x7f00040a9b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f001c10d690 tx=0x7f00100095d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.000+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f000c0bd530 con 0x7f001c102d40 2024-09-15T23:57:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.233+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f001c1071e0 con 0x7f001c102d40 2024-09-15T23:57:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.234+0000 7f0003fff640 1 -- 172.21.15.31:0/1231336570 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f000c0bce30 con 0x7f001c102d40 2024-09-15T23:57:36.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:36.235 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"btime":"2024-09-15T23:57:36:012153+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T23:57:36.235 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-09-15T23:57:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.236+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 msgr2=0x7f00040a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.236+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 0x7f00040a9b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f001c10d690 tx=0x7f00100095d0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.236+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 msgr2=0x7f001c196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.236+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f000c031440 tx=0x7f000c003cd0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 shutdown_connections 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f00040a7690 0x7f00040a9b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001c10c170 0x7f001c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f001c103660 0x7f001c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 --2- 172.21.15.31:0/1231336570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f001c102d40 0x7f001c196160 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 >> 172.21.15.31:0/1231336570 conn(0x7f001c0fc7a0 msgr2=0x7f001c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 shutdown_connections 2024-09-15T23:57:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:36.237+0000 7f0021599640 1 -- 172.21.15.31:0/1231336570 wait complete. 2024-09-15T23:57:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: Deploying daemon mds.d on smithi031 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6834/3861052716,v1:172.21.15.31:6835/3861052716] up:boot 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: fsmap 3 up:standby 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T23:57:36.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:36 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1231336570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:36.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:36.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: Deploying daemon mds.d on smithi031 2024-09-15T23:57:36.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6834/3861052716,v1:172.21.15.31:6835/3861052716] up:boot 2024-09-15T23:57:36.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: fsmap 3 up:standby 2024-09-15T23:57:36.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T23:57:36.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:36 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1231336570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: Deploying daemon mds.d on smithi031 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6834/3861052716,v1:172.21.15.31:6835/3861052716] up:boot 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: fsmap 3 up:standby 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T23:57:36.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:36 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1231336570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:38.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:38 smithi031 ceph-mon[31203]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:38.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:38 smithi081 ceph-mon[35060]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:38.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:38 smithi102 ceph-mon[34458]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:38.902 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 15 23:57:38 smithi031 podman[71399]: 2024-09-15 23:57:38.878241783 +0000 UTC m=+0.362589676 container start f61cf5472edb947dfcf7390cbc89b7ea1f99d6ca6c00f539229911448f2dffd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T23:57:39.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:39.254 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 15 23:57:38 smithi031 ceph-mds[71418]: -- 172.21.15.31:0/353951713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f818871860 con 0x55f81884e800 2024-09-15T23:57:39.254 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 15 23:57:38 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d[71414]: starting mds.d at 2024-09-15T23:57:39.254 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 15 23:57:38 smithi031 bash[71399]: f61cf5472edb947dfcf7390cbc89b7ea1f99d6ca6c00f539229911448f2dffd9 2024-09-15T23:57:39.254 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 15 23:57:38 smithi031 systemd[1]: Started Ceph mds.d for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:39.299 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 -- 172.21.15.31:0/3150875691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728100ca0 msgr2=0x7fd728106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/3150875691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728100ca0 0x7fd728106840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd710009930 tx=0x7fd71002a420 comp rx=0 tx=0).stop 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 -- 172.21.15.31:0/3150875691 shutdown_connections 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/3150875691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728106f50 0x7fd72810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/3150875691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728100ca0 0x7fd728106840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/3150875691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 0x7fd7281006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 -- 172.21.15.31:0/3150875691 >> 172.21.15.31:0/3150875691 conn(0x7fd728077210 msgr2=0x7fd7280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 -- 172.21.15.31:0/3150875691 shutdown_connections 2024-09-15T23:57:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.778+0000 7fd72d0c4640 1 -- 172.21.15.31:0/3150875691 wait complete. 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 Processor -- start 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 -- start start 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 0x7fd728101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 0x7fd7281017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd728111e50 con 0x7fd728106f50 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd728111cd0 con 0x7fd728100ca0 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72d0c4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd728111fd0 con 0x7fd7281002f0 2024-09-15T23:57:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36812/0 (socket says 172.21.15.31:36812) 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd72759e640 1 -- 172.21.15.31:0/271927957 learned_addr learned my addr 172.21.15.31:0/271927957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.779+0000 7fd726d9d640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 0x7fd728101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72659c640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 0x7fd7281017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 -- 172.21.15.31:0/271927957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 msgr2=0x7fd728101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 0x7fd728101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 -- 172.21.15.31:0/271927957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 msgr2=0x7fd7281017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 0x7fd7281017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 -- 172.21.15.31:0/271927957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7281a6950 con 0x7fd728106f50 2024-09-15T23:57:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72659c640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 0x7fd7281017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72759e640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd72000eaa0 tx=0x7fd72000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72000cdb0 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7281a6c40 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd720005020 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200116d0 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.780+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7281a7100 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.781+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7281006d0 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.782+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd7200051d0 con 0x7fd728106f50 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.784+0000 7fd71ffff640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 0x7fd7140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.784+0000 7fd726d9d640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 0x7fd7140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.784+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd7200cf600 con 0x7fd728106f50 2024-09-15T23:57:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.784+0000 7fd726d9d640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 0x7fd7140a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd718009bd0 tx=0x7fd718005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.788+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd720016030 con 0x7fd728106f50 2024-09-15T23:57:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.998+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd72810a120 con 0x7fd728106f50 2024-09-15T23:57:40.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:39.999+0000 7fd71ffff640 1 -- 172.21.15.31:0/271927957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7fd72009c7c0 con 0x7fd728106f50 2024-09-15T23:57:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:40.001 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"btime":"2024-09-15T23:57:39:414189+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T23:57:40.001 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 msgr2=0x7fd7140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 0x7fd7140a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd718009bd0 tx=0x7fd718005f90 comp rx=0 tx=0).stop 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 msgr2=0x7fd728101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd72000eaa0 tx=0x7fd72000ef60 comp rx=0 tx=0).stop 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 shutdown_connections 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7140a3240 0x7fd7140a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728106f50 0x7fd728101cf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd728100ca0 0x7fd7281017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:40.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 --2- 172.21.15.31:0/271927957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7281002f0 0x7fd728101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 >> 172.21.15.31:0/271927957 conn(0x7fd728077210 msgr2=0x7fd7281052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 shutdown_connections 2024-09-15T23:57:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:40.001+0000 7fd72d0c4640 1 -- 172.21.15.31:0/271927957 wait complete. 2024-09-15T23:57:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: Deploying daemon mds.e on smithi081 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6836/735300624,v1:172.21.15.31:6837/735300624] up:boot 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: fsmap 4 up:standby 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T23:57:40.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:40 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/271927957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: Deploying daemon mds.e on smithi081 2024-09-15T23:57:40.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6836/735300624,v1:172.21.15.31:6837/735300624] up:boot 2024-09-15T23:57:40.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: fsmap 4 up:standby 2024-09-15T23:57:40.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T23:57:40.308 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:40 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/271927957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:40.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: Deploying daemon mds.e on smithi081 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6836/735300624,v1:172.21.15.31:6837/735300624] up:boot 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: fsmap 4 up:standby 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T23:57:40.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:40 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/271927957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:41.266 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:41 smithi081 ceph-mon[35060]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:41.266 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 systemd[1]: Starting Ceph mds.e for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:41 smithi102 ceph-mon[34458]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:41 smithi031 ceph-mon[31203]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:41.526 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 podman[64095]: 2024-09-15 23:57:41.168279428 +0000 UTC m=+0.016153954 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:41.526 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 podman[64095]: 2024-09-15 23:57:41.289496949 +0000 UTC m=+0.137371465 container create 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:57:41.526 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 podman[64095]: 2024-09-15 23:57:41.525788433 +0000 UTC m=+0.373662948 container init 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-15T23:57:41.807 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 podman[64095]: 2024-09-15 23:57:41.528805517 +0000 UTC m=+0.376680032 container start 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:57:41.807 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 ceph-mds[64114]: -- 172.21.15.81:0/608380671 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aacc71b860 con 0x55aacc6f9000 2024-09-15T23:57:41.807 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: starting mds.e at 2024-09-15T23:57:41.807 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 bash[64095]: 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 2024-09-15T23:57:41.807 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 15 23:57:41 smithi081 systemd[1]: Started Ceph mds.e for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:42.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:42.983 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:43.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: Deploying daemon mds.f on smithi102 2024-09-15T23:57:43.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6834/799148008,v1:172.21.15.81:6835/799148008] up:boot 2024-09-15T23:57:43.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: fsmap 5 up:standby 2024-09-15T23:57:43.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:42 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: Deploying daemon mds.f on smithi102 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6834/799148008,v1:172.21.15.81:6835/799148008] up:boot 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: fsmap 5 up:standby 2024-09-15T23:57:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:42 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:57:43.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: Deploying daemon mds.f on smithi102 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6834/799148008,v1:172.21.15.81:6835/799148008] up:boot 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: fsmap 5 up:standby 2024-09-15T23:57:43.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:42 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 -- 172.21.15.31:0/282314107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 msgr2=0x7fda44107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/282314107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44107b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda38009930 tx=0x7fda3802a420 comp rx=0 tx=0).stop 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 -- 172.21.15.31:0/282314107 shutdown_connections 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/282314107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 0x7fda4410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/282314107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44107b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/282314107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda44103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 -- 172.21.15.31:0/282314107 >> 172.21.15.31:0/282314107 conn(0x7fda440fc7d0 msgr2=0x7fda440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 -- 172.21.15.31:0/282314107 shutdown_connections 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.630+0000 7fda4a6b3640 1 -- 172.21.15.31:0/282314107 wait complete. 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 Processor -- start 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 -- start start 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda441985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 0x7fda4419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.631+0000 7fda4a6b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111f70 con 0x7fda441036e0 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda4a6b3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111df0 con 0x7fda441082a0 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda4a6b3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fda441120f0 con 0x7fda44102d30 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda43fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda441985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42270/0 (socket says 172.21.15.31:42270) 2024-09-15T23:57:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 -- 172.21.15.31:0/2055289808 learned_addr learned my addr 172.21.15.31:0/2055289808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda48c50640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 0x7fda4419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 -- 172.21.15.31:0/2055289808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 msgr2=0x7fda441985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda441985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 -- 172.21.15.31:0/2055289808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 msgr2=0x7fda4419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 0x7fda4419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 -- 172.21.15.31:0/2055289808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4419d560 con 0x7fda441036e0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda43fff640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda437fe640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda38031440 tx=0x7fda38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3803f030 con 0x7fda441036e0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4419d7f0 con 0x7fda441036e0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda38003aa0 con 0x7fda441036e0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.632+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38035070 con 0x7fda441036e0 2024-09-15T23:57:43.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.633+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441a5130 con 0x7fda441036e0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.634+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda44103110 con 0x7fda441036e0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.635+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fda38042030 con 0x7fda441036e0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.635+0000 7fda417fa640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 0x7fda2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.636+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fda380eff40 con 0x7fda441036e0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.636+0000 7fda43fff640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 0x7fda2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.636+0000 7fda43fff640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 0x7fda2c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda34009d60 tx=0x7fda34009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:43.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.639+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda380bd100 con 0x7fda441036e0 2024-09-15T23:57:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.853+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda44199910 con 0x7fda441036e0 2024-09-15T23:57:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.853+0000 7fda417fa640 1 -- 172.21.15.31:0/2055289808 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7fda44199910 con 0x7fda441036e0 2024-09-15T23:57:43.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:43.855 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"btime":"2024-09-15T23:57:42:047507+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-15T23:57:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-09-15T23:57:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 msgr2=0x7fda2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 0x7fda2c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda34009d60 tx=0x7fda34009110 comp rx=0 tx=0).stop 2024-09-15T23:57:43.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 msgr2=0x7fda44198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:43.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda38031440 tx=0x7fda38004270 comp rx=0 tx=0).stop 2024-09-15T23:57:43.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 shutdown_connections 2024-09-15T23:57:43.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fda2c0a3240 0x7fda2c0a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda441082a0 0x7fda4419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda441036e0 0x7fda44198af0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 --2- 172.21.15.31:0/2055289808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda44102d30 0x7fda441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:43.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 >> 172.21.15.31:0/2055289808 conn(0x7fda440fc7d0 msgr2=0x7fda44105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:43.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 shutdown_connections 2024-09-15T23:57:43.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:43.856+0000 7fda4a6b3640 1 -- 172.21.15.31:0/2055289808 wait complete. 2024-09-15T23:57:43.870 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:43 smithi102 systemd[1]: Starting Ceph mds.f for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:44.228 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:43 smithi102 podman[64147]: 2024-09-15 23:57:43.841516126 +0000 UTC m=+0.015480349 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:44.228 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:43 smithi102 podman[64147]: 2024-09-15 23:57:43.966238622 +0000 UTC m=+0.140202835 container create c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T23:57:44.228 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 podman[64147]: 2024-09-15 23:57:44.227244535 +0000 UTC m=+0.401208750 container init c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:57:44.620 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 podman[64147]: 2024-09-15 23:57:44.230196085 +0000 UTC m=+0.404160308 container start c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-15T23:57:44.621 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mds[64166]: -- 172.21.15.102:0/636283828 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e813f11860 con 0x55e813eee800 2024-09-15T23:57:44.621 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: starting mds.f at 2024-09-15T23:57:44.621 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 bash[64147]: c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c 2024-09-15T23:57:44.621 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 15 23:57:44 smithi102 systemd[1]: Started Ceph mds.f for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2055289808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:44.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:44 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2055289808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:45.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:45.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:44 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2055289808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:44 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:45.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:45 smithi102 ceph-mon[34458]: Deploying daemon mds.c on smithi102 2024-09-15T23:57:45.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:45 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:boot 2024-09-15T23:57:45.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:45 smithi102 ceph-mon[34458]: fsmap 6 up:standby 2024-09-15T23:57:45.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:45 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:57:46.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:45 smithi031 ceph-mon[31203]: Deploying daemon mds.c on smithi102 2024-09-15T23:57:46.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:45 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:boot 2024-09-15T23:57:46.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:45 smithi031 ceph-mon[31203]: fsmap 6 up:standby 2024-09-15T23:57:46.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:45 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:57:46.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:45 smithi081 ceph-mon[35060]: Deploying daemon mds.c on smithi102 2024-09-15T23:57:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:45 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:boot 2024-09-15T23:57:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:45 smithi081 ceph-mon[35060]: fsmap 6 up:standby 2024-09-15T23:57:46.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:45 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:57:46.578 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 systemd[1]: Starting Ceph mds.c for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:46.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:46.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:46 smithi102 ceph-mon[34458]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:46.851 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 podman[64446]: 2024-09-15 23:57:46.479818092 +0000 UTC m=+0.016027598 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:46.851 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 podman[64446]: 2024-09-15 23:57:46.589143898 +0000 UTC m=+0.125353394 container create 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:57:46.851 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 podman[64446]: 2024-09-15 23:57:46.850540642 +0000 UTC m=+0.386750140 container init 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T23:57:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:46.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:46 smithi031 ceph-mon[31203]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:47.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:46 smithi081 ceph-mon[35060]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:47.121 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 podman[64446]: 2024-09-15 23:57:46.853725325 +0000 UTC m=+0.389934818 container start 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:57:47.121 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 ceph-mds[64466]: -- 172.21.15.102:0/4102924135 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae57d51860 con 0x55ae57d2e800 2024-09-15T23:57:47.121 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: starting mds.c at 2024-09-15T23:57:47.121 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 bash[64446]: 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f 2024-09-15T23:57:47.121 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 15 23:57:46 smithi102 systemd[1]: Started Ceph mds.c for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 -- 172.21.15.31:0/3821219187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c106f10 msgr2=0x7f662c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 --2- 172.21.15.31:0/3821219187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c106f10 0x7f662c10ea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f661c009ca0 tx=0x7f661c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 -- 172.21.15.31:0/3821219187 shutdown_connections 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 --2- 172.21.15.31:0/3821219187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c106f10 0x7f662c10ea90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 --2- 172.21.15.31:0/3821219187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c102790 0x7f662c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 --2- 172.21.15.31:0/3821219187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c101de0 0x7f662c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 -- 172.21.15.31:0/3821219187 >> 172.21.15.31:0/3821219187 conn(0x7f662c0fd8c0 msgr2=0x7f662c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 -- 172.21.15.31:0/3821219187 shutdown_connections 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 -- 172.21.15.31:0/3821219187 wait complete. 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.313+0000 7f6633f16640 1 Processor -- start 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 -- start start 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c101de0 0x7f662c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c106f10 0x7f662c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f662c111e60 con 0x7f662c102790 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f662c111ce0 con 0x7f662c106f10 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6633f16640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f662c111fe0 con 0x7f662c101de0 2024-09-15T23:57:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42290/0 (socket says 172.21.15.31:42290) 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 -- 172.21.15.31:0/4088194823 learned_addr learned my addr 172.21.15.31:0/4088194823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66324b3640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c106f10 0x7f662c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f6631cb2640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c101de0 0x7f662c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 -- 172.21.15.31:0/4088194823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c101de0 msgr2=0x7f662c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c101de0 0x7f662c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 -- 172.21.15.31:0/4088194823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c106f10 msgr2=0x7f662c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c106f10 0x7f662c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 -- 172.21.15.31:0/4088194823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c19d3d0 con 0x7f662c102790 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.314+0000 7f66314b1640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f662000d770 tx=0x7f662000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.315+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620017070 con 0x7f662c102790 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.315+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6620009820 con 0x7f662c102790 2024-09-15T23:57:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.315+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620004ba0 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.315+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662c19d6c0 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.315+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662c1a4fa0 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.316+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662c1021c0 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.317+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f66200099d0 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.317+0000 7f662affd640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 0x7f66140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.317+0000 7f6631cb2640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 0x7f66140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.318+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f66200cee60 con 0x7f662c102790 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.318+0000 7f6631cb2640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 0x7f66140a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6624009990 tx=0x7f6624009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.322+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f662009c020 con 0x7f662c102790 2024-09-15T23:57:47.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.535+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f662c10a0f0 con 0x7f662c102790 2024-09-15T23:57:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.535+0000 7f662affd640 1 -- 172.21.15.31:0/4088194823 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5557 (secure 0 0 0) 0x7f662009b920 con 0x7f662c102790 2024-09-15T23:57:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:47.538 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"btime":"2024-09-15T23:57:44:676427+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-15T23:57:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-09-15T23:57:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 msgr2=0x7f66140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 0x7f66140a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6624009990 tx=0x7f6624009110 comp rx=0 tx=0).stop 2024-09-15T23:57:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 msgr2=0x7f662c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f662000d770 tx=0x7f662000db70 comp rx=0 tx=0).stop 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 shutdown_connections 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f66140a3240 0x7f66140a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f662c106f10 0x7f662c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f662c102790 0x7f662c198960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 --2- 172.21.15.31:0/4088194823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662c101de0 0x7f662c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 >> 172.21.15.31:0/4088194823 conn(0x7f662c0fd8c0 msgr2=0x7f662c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.537+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 shutdown_connections 2024-09-15T23:57:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:47.538+0000 7f6633f16640 1 -- 172.21.15.31:0/4088194823 wait complete. 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: Deploying daemon mds.h on smithi081 2024-09-15T23:57:48.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4088194823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:48.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:boot 2024-09-15T23:57:48.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: fsmap 7 up:standby 2024-09-15T23:57:48.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:48 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:57:48.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:48.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: Deploying daemon mds.h on smithi081 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4088194823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:boot 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: fsmap 7 up:standby 2024-09-15T23:57:48.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:48 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:57:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: Deploying daemon mds.h on smithi081 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4088194823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:boot 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: fsmap 7 up:standby 2024-09-15T23:57:48.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:48 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:57:49.235 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:49 smithi081 ceph-mon[35060]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:49.235 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 systemd[1]: Starting Ceph mds.h for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:49 smithi102 ceph-mon[34458]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:49.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:49 smithi031 ceph-mon[31203]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:49.557 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 podman[64396]: 2024-09-15 23:57:49.137413785 +0000 UTC m=+0.016411040 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:49.557 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 podman[64396]: 2024-09-15 23:57:49.254476808 +0000 UTC m=+0.133474063 container create fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 podman[64396]: 2024-09-15 23:57:49.557777669 +0000 UTC m=+0.436774927 container init fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 podman[64396]: 2024-09-15 23:57:49.560585447 +0000 UTC m=+0.439582711 container start fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 ceph-mds[64415]: -- 172.21.15.81:0/105575014 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558325df9860 con 0x558325dd7400 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: starting mds.h at 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 bash[64396]: fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec 2024-09-15T23:57:50.057 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 15 23:57:49 smithi081 systemd[1]: Started Ceph mds.h for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:50.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:50.549 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:50.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: Deploying daemon mds.i on smithi102 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:boot 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: fsmap 8 up:standby 2024-09-15T23:57:50.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:50 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:57:51.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:51.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:51.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: Deploying daemon mds.i on smithi102 2024-09-15T23:57:51.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:boot 2024-09-15T23:57:51.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: fsmap 8 up:standby 2024-09-15T23:57:51.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:50 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:57:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- 172.21.15.31:0/2851088685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 msgr2=0x7f491410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 --2- 172.21.15.31:0/2851088685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f491410ea90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f490c009ca0 tx=0x7f490c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- 172.21.15.31:0/2851088685 shutdown_connections 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 --2- 172.21.15.31:0/2851088685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f491410ea90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 --2- 172.21.15.31:0/2851088685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914102790 0x7f49141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 --2- 172.21.15.31:0/2851088685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914101de0 0x7f49141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- 172.21.15.31:0/2851088685 >> 172.21.15.31:0/2851088685 conn(0x7f49140fd8c0 msgr2=0x7f49140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- 172.21.15.31:0/2851088685 shutdown_connections 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- 172.21.15.31:0/2851088685 wait complete. 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 Processor -- start 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.054+0000 7f4919a18640 1 -- start start 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914101de0 0x7f49140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 0x7f4914074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4914111e20 con 0x7f4914106f10 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4914111ca0 con 0x7f4914102790 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4919a18640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4914111fa0 con 0x7f4914101de0 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42312/0 (socket says 172.21.15.31:42312) 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 -- 172.21.15.31:0/4085387217 learned_addr learned my addr 172.21.15.31:0/4085387217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f4912ffd640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914101de0 0x7f49140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f490a7fc640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 0x7f4914074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 -- 172.21.15.31:0/4085387217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914101de0 msgr2=0x7f49140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914101de0 0x7f49140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 -- 172.21.15.31:0/4085387217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 msgr2=0x7f4914074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 0x7f4914074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 -- 172.21.15.31:0/4085387217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49141aac50 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f490a7fc640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 0x7f4914074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.055+0000 7f49137fe640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f490c009dc0 tx=0x7f490c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.056+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c003bf0 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.056+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f490c003d90 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.056+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c030ac0 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.056+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49141aae80 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.056+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141ab2e0 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.057+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49141032c0 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.058+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f490c004610 con 0x7f4914106f10 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.059+0000 7f4918a16640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 0x7f49000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.059+0000 7f4912ffd640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 0x7f49000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.059+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f490c0f09e0 con 0x7f4914106f10 2024-09-15T23:57:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.059+0000 7f4912ffd640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 0x7f49000a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f490400abe0 tx=0x7f490400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.063+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f490c0bdcc0 con 0x7f4914106f10 2024-09-15T23:57:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: Deploying daemon mds.i on smithi102 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:boot 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: fsmap 8 up:standby 2024-09-15T23:57:51.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:50 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:57:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.274+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4914075a20 con 0x7f4914106f10 2024-09-15T23:57:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.275+0000 7f4918a16640 1 -- 172.21.15.31:0/4085387217 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f490c0bd5c0 con 0x7f4914106f10 2024-09-15T23:57:51.277 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:51.277 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"btime":"2024-09-15T23:57:50:016421+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T23:57:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 msgr2=0x7f49000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 0x7f49000a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f490400abe0 tx=0x7f490400a2d0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 msgr2=0x7f4914074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f490c009dc0 tx=0x7f490c032ff0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 shutdown_connections 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49000a32c0 0x7f49000a5780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4914106f10 0x7f4914074f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4914102790 0x7f4914074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 --2- 172.21.15.31:0/4085387217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4914101de0 0x7f49140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 >> 172.21.15.31:0/4085387217 conn(0x7f49140fd8c0 msgr2=0x7f49140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 shutdown_connections 2024-09-15T23:57:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:51.277+0000 7f4919a18640 1 -- 172.21.15.31:0/4085387217 wait complete. 2024-09-15T23:57:51.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:51 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4085387217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:51.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:51 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4085387217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:52.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:51 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4085387217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:52.620 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 15 23:57:52 smithi102 systemd[1]: Started Ceph mds.i for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:53.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:53.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:52 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:53.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:52 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:53.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:53.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:52 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:54.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:54.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:53 smithi031 ceph-mon[31203]: Deploying daemon mds.g on smithi031 2024-09-15T23:57:54.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:boot 2024-09-15T23:57:54.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:53 smithi031 ceph-mon[31203]: fsmap 9 up:standby 2024-09-15T23:57:54.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:53 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:57:54.022 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:53 smithi102 ceph-mon[34458]: Deploying daemon mds.g on smithi031 2024-09-15T23:57:54.022 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:boot 2024-09-15T23:57:54.022 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:53 smithi102 ceph-mon[34458]: fsmap 9 up:standby 2024-09-15T23:57:54.022 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:53 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:57:54.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:53 smithi081 ceph-mon[35060]: Deploying daemon mds.g on smithi031 2024-09-15T23:57:54.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:boot 2024-09-15T23:57:54.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:53 smithi081 ceph-mon[35060]: fsmap 9 up:standby 2024-09-15T23:57:54.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:53 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:57:54.374 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:54.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 -- 172.21.15.31:0/2031124479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101de0 msgr2=0x7f9f0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/2031124479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101de0 0x7f9f0c1021c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009900 tx=0x7f9efc02a410 comp rx=0 tx=0).stop 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 -- 172.21.15.31:0/2031124479 shutdown_connections 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/2031124479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c106f10 0x7f9f0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/2031124479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c102790 0x7f9f0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/2031124479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c101de0 0x7f9f0c1021c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 -- 172.21.15.31:0/2031124479 >> 172.21.15.31:0/2031124479 conn(0x7f9f0c0fd8c0 msgr2=0x7f9f0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 -- 172.21.15.31:0/2031124479 shutdown_connections 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.892+0000 7f9f10fa0640 1 -- 172.21.15.31:0/2031124479 wait complete. 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 Processor -- start 2024-09-15T23:57:54.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 -- start start 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c102790 0x7f9f0c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 0x7f9f0c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c111e20 con 0x7f9f0c106f10 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c111ca0 con 0x7f9f0c101de0 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f10fa0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c111fa0 con 0x7f9f0c102790 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f0ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 0x7f9f0c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:54.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:46378/0 (socket says 172.21.15.31:46378) 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.893+0000 7f9f0a59c640 1 -- 172.21.15.31:0/1410084810 learned_addr learned my addr 172.21.15.31:0/1410084810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f09d9b640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c102790 0x7f9f0c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 -- 172.21.15.31:0/1410084810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c102790 msgr2=0x7f9f0c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c102790 0x7f9f0c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 -- 172.21.15.31:0/1410084810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 msgr2=0x7f9f0c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 -- 172.21.15.31:0/1410084810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c19d310 con 0x7f9f0c106f10 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0a59c640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9f0ad9d640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 0x7f9f0c19cc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009fd0 tx=0x7f9f0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400ce20 con 0x7f9f0c106f10 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.894+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f04005020 con 0x7f9f0c106f10 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.895+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c19d600 con 0x7f9f0c106f10 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.895+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c1a4ee0 con 0x7f9f0c106f10 2024-09-15T23:57:54.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.895+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04010690 con 0x7f9f0c106f10 2024-09-15T23:57:54.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.896+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f0c1021c0 con 0x7f9f0c106f10 2024-09-15T23:57:54.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.897+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9f040051d0 con 0x7f9f0c106f10 2024-09-15T23:57:54.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.898+0000 7f9efb7fe640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 0x7f9ef40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:54.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.898+0000 7f9f0a59c640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 0x7f9ef40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:54.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.898+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9f040cfa50 con 0x7f9f0c106f10 2024-09-15T23:57:54.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.898+0000 7f9f0a59c640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 0x7f9ef40a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9efc031440 tx=0x7f9efc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:54.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:54.905+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f0409cb70 con 0x7f9f0c106f10 2024-09-15T23:57:54.906 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:54 smithi031 ceph-mon[31203]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:55.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:54 smithi081 ceph-mon[35060]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:55.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.119+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f0c02d7c0 con 0x7f9f0c106f10 2024-09-15T23:57:55.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.119+0000 7f9efb7fe640 1 -- 172.21.15.31:0/1410084810 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8040 (secure 0 0 0) 0x7f9f0409c470 con 0x7f9f0c106f10 2024-09-15T23:57:55.120 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:55.120 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"btime":"2024-09-15T23:57:52:712816+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-15T23:57:55.121 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-09-15T23:57:55.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:54 smithi102 ceph-mon[34458]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 msgr2=0x7f9ef40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 0x7f9ef40a5780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9efc031440 tx=0x7f9efc0058d0 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 msgr2=0x7f9f0c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 0x7f9f0c19cc30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009fd0 tx=0x7f9f0400ef10 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 shutdown_connections 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ef40a32c0 0x7f9ef40a5780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0c106f10 0x7f9f0c19cc30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c102790 0x7f9f0c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 --2- 172.21.15.31:0/1410084810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f0c101de0 0x7f9f0c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 >> 172.21.15.31:0/1410084810 conn(0x7f9f0c0fd8c0 msgr2=0x7f9f0c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 shutdown_connections 2024-09-15T23:57:55.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:55.122+0000 7f9f10fa0640 1 -- 172.21.15.31:0/1410084810 wait complete. 2024-09-15T23:57:55.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1410084810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:55.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:55.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:55.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:55.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:55.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:55.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:55 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1410084810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:56.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:55 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:56.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1410084810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:57:56.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:56.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:56.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:56.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:55 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:57.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:56 smithi031 ceph-mon[31203]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:57.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:56 smithi031 ceph-mon[31203]: Deploying daemon mds.a on smithi031 2024-09-15T23:57:57.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:56 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6838/193010383,v1:172.21.15.31:6839/193010383] up:boot 2024-09-15T23:57:57.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:56 smithi031 ceph-mon[31203]: fsmap 10 up:standby 2024-09-15T23:57:57.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:56 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T23:57:57.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:56 smithi081 ceph-mon[35060]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:57.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:56 smithi081 ceph-mon[35060]: Deploying daemon mds.a on smithi031 2024-09-15T23:57:57.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:56 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6838/193010383,v1:172.21.15.31:6839/193010383] up:boot 2024-09-15T23:57:57.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:56 smithi081 ceph-mon[35060]: fsmap 10 up:standby 2024-09-15T23:57:57.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:56 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T23:57:57.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:56 smithi102 ceph-mon[34458]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:57.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:56 smithi102 ceph-mon[34458]: Deploying daemon mds.a on smithi031 2024-09-15T23:57:57.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:56 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6838/193010383,v1:172.21.15.31:6839/193010383] up:boot 2024-09-15T23:57:57.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:56 smithi102 ceph-mon[34458]: fsmap 10 up:standby 2024-09-15T23:57:57.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:56 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T23:57:58.004 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:57 smithi031 systemd[1]: Starting Ceph mds.a for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:57:58.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:57:58.339 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 podman[72587]: 2024-09-15 23:57:57.945586968 +0000 UTC m=+0.016561639 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:57:58.339 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 podman[72587]: 2024-09-15 23:57:58.077335647 +0000 UTC m=+0.148310346 container create 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:57:58.530 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 podman[72587]: 2024-09-15 23:57:58.33879667 +0000 UTC m=+0.409771330 container init 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 podman[72587]: 2024-09-15 23:57:58.342148972 +0000 UTC m=+0.413123630 container start 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mds[72631]: -- 172.21.15.31:0/1812759258 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c55484f860 con 0x55c55482d400 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: starting mds.a at 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 bash[72587]: 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 2024-09-15T23:57:58.653 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 15 23:57:58 smithi031 systemd[1]: Started Ceph mds.a for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:57:58.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:58.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:58 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:59.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:58 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 -- 172.21.15.31:0/2881598765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 msgr2=0x7f30f010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 --2- 172.21.15.31:0/2881598765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f010e610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009ca0 tx=0x7f30e802a950 comp rx=0 tx=0).stop 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 -- 172.21.15.31:0/2881598765 shutdown_connections 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 --2- 172.21.15.31:0/2881598765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f010e610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 --2- 172.21.15.31:0/2881598765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0103660 0x7f30f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 --2- 172.21.15.31:0/2881598765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0102d40 0x7f30f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 -- 172.21.15.31:0/2881598765 >> 172.21.15.31:0/2881598765 conn(0x7f30f00fc7a0 msgr2=0x7f30f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 -- 172.21.15.31:0/2881598765 shutdown_connections 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.067+0000 7f30f5e09640 1 -- 172.21.15.31:0/2881598765 wait complete. 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 Processor -- start 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 -- start start 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0102d40 0x7f30f0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0103660 0x7f30f0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111de0 con 0x7f30f010c170 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111c60 con 0x7f30f0103660 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30f5e09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111f60 con 0x7f30f0102d40 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47946/0 (socket says 172.21.15.31:47946) 2024-09-15T23:57:59.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30effff640 1 -- 172.21.15.31:0/1998351659 learned_addr learned my addr 172.21.15.31:0/1998351659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.068+0000 7f30eeffd640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0103660 0x7f30f0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30ef7fe640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0102d40 0x7f30f0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 -- 172.21.15.31:0/1998351659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0102d40 msgr2=0x7f30f0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0102d40 0x7f30f0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 -- 172.21.15.31:0/1998351659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0103660 msgr2=0x7f30f0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0103660 0x7f30f0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 -- 172.21.15.31:0/1998351659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f019d2d0 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30effff640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30e8004af0 tx=0x7f30e802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8034040 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f019d560 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e802fe80 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80046a0 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.069+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01a4ea0 con 0x7f30f010c170 2024-09-15T23:57:59.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.070+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f0103120 con 0x7f30f010c170 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.071+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f30e8037020 con 0x7f30f010c170 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.072+0000 7f30ecff9640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 0x7f30d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.072+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f30e80f02d0 con 0x7f30f010c170 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.072+0000 7f30ef7fe640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 0x7f30d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.073+0000 7f30ef7fe640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 0x7f30d80a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009bd0 tx=0x7f30e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:57:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.075+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30e80bd490 con 0x7f30f010c170 2024-09-15T23:57:59.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:57:59.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:57:59.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T23:57:59.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T23:57:59.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:58 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:57:59.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.288+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f01071e0 con 0x7f30f010c170 2024-09-15T23:57:59.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.289+0000 7f30ecff9640 1 -- 172.21.15.31:0/1998351659 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7f30e80bcd90 con 0x7f30f010c170 2024-09-15T23:57:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:57:59.290 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"btime":"2024-09-15T23:57:58:730649+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-15T23:57:59.290 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-09-15T23:57:59.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 msgr2=0x7f30d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 0x7f30d80a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009bd0 tx=0x7f30e0005f90 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 msgr2=0x7f30f019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30e8004af0 tx=0x7f30e802f5d0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 shutdown_connections 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f30d80a3240 0x7f30d80a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f010c170 0x7f30f019cbf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30f0103660 0x7f30f0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 --2- 172.21.15.31:0/1998351659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30f0102d40 0x7f30f0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 >> 172.21.15.31:0/1998351659 conn(0x7f30f00fc7a0 msgr2=0x7f30f00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 shutdown_connections 2024-09-15T23:57:59.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:57:59.291+0000 7f30f5e09640 1 -- 172.21.15.31:0/1998351659 wait complete. 2024-09-15T23:57:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:59 smithi031 ceph-mon[31203]: Deploying daemon mds.b on smithi081 2024-09-15T23:57:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:59 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6840/1341414618,v1:172.21.15.31:6841/1341414618] up:boot 2024-09-15T23:57:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:59 smithi031 ceph-mon[31203]: fsmap 11 up:standby 2024-09-15T23:57:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:59 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T23:57:59.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:57:59 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1998351659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:00.037 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:59 smithi081 ceph-mon[35060]: Deploying daemon mds.b on smithi081 2024-09-15T23:58:00.037 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:59 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6840/1341414618,v1:172.21.15.31:6841/1341414618] up:boot 2024-09-15T23:58:00.037 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:59 smithi081 ceph-mon[35060]: fsmap 11 up:standby 2024-09-15T23:58:00.037 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:59 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T23:58:00.037 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:57:59 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1998351659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:00.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:59 smithi102 ceph-mon[34458]: Deploying daemon mds.b on smithi081 2024-09-15T23:58:00.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:59 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6840/1341414618,v1:172.21.15.31:6841/1341414618] up:boot 2024-09-15T23:58:00.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:59 smithi102 ceph-mon[34458]: fsmap 11 up:standby 2024-09-15T23:58:00.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:59 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T23:58:00.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:57:59 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1998351659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:01.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:00 smithi031 ceph-mon[31203]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:01.011 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:00 smithi081 ceph-mon[35060]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:01.012 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:00 smithi081 systemd[1]: Starting Ceph mds.b for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-15T23:58:01.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:00 smithi102 ceph-mon[34458]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:01.307 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 podman[64712]: 2024-09-15 23:58:00.9137937 +0000 UTC m=+0.016098582 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:58:01.307 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 podman[64712]: 2024-09-15 23:58:01.036766496 +0000 UTC m=+0.139071382 container create 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 podman[64712]: 2024-09-15 23:58:01.323644284 +0000 UTC m=+0.425949166 container init 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 podman[64712]: 2024-09-15 23:58:01.32667111 +0000 UTC m=+0.428976000 container start 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 ceph-mds[64733]: -- 172.21.15.81:0/557522444 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c05dee7860 con 0x55c05dec4800 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: starting mds.b at 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 bash[64712]: 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd 2024-09-15T23:58:01.807 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:01 smithi081 systemd[1]: Started Ceph mds.b for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-15T23:58:01.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:58:02.486 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.487 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:02.488 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:boot 2024-09-15T23:58:02.488 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: fsmap 12 up:standby 2024-09-15T23:58:02.488 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:02 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:boot 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: fsmap 12 up:standby 2024-09-15T23:58:02.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:02 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:58:02.720 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:02.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:02.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:boot 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: fsmap 12 up:standby 2024-09-15T23:58:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:02 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 -- 172.21.15.31:0/1461717232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc100e20 msgr2=0x7fbdcc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/1461717232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc100e20 0x7fbdcc101200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009900 tx=0x7fbdc802a480 comp rx=0 tx=0).stop 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 -- 172.21.15.31:0/1461717232 shutdown_connections 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/1461717232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/1461717232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc102570 0x7fbdcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/1461717232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc100e20 0x7fbdcc101200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 -- 172.21.15.31:0/1461717232 >> 172.21.15.31:0/1461717232 conn(0x7fbdcc0fc7d0 msgr2=0x7fbdcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 -- 172.21.15.31:0/1461717232 shutdown_connections 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.254+0000 7fbdd3a16640 1 -- 172.21.15.31:0/1461717232 wait complete. 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 Processor -- start 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 -- start start 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc100e20 0x7fbdcc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102570 0x7fbdcc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc110e80 con 0x7fbdcc102ef0 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc110d00 con 0x7fbdcc102570 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd3a16640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc111000 con 0x7fbdcc100e20 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57108/0 (socket says 172.21.15.31:57108) 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 -- 172.21.15.31:0/112543465 learned_addr learned my addr 172.21.15.31:0/112543465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd17b2640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc100e20 0x7fbdcc103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd0fb1640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102570 0x7fbdcc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 -- 172.21.15.31:0/112543465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc100e20 msgr2=0x7fbdcc103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc100e20 0x7fbdcc103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 -- 172.21.15.31:0/112543465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102570 msgr2=0x7fbdcc103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102570 0x7fbdcc103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.255+0000 7fbdd1fb3640 1 -- 172.21.15.31:0/112543465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc1a68c0 con 0x7fbdcc102ef0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdd1fb3640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fd0 tx=0x7fbdc400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4017070 con 0x7fbdcc102ef0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc4003e20 con 0x7fbdcc102ef0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc1a6bb0 con 0x7fbdcc102ef0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4010460 con 0x7fbdcc102ef0 2024-09-15T23:58:03.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.256+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc1a70a0 con 0x7fbdcc102ef0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.257+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc101200 con 0x7fbdcc102ef0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.258+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbdc4003970 con 0x7fbdcc102ef0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.259+0000 7fbdc27fc640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 0x7fbdb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.259+0000 7fbdd17b2640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 0x7fbdb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.259+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbdc40cf1c0 con 0x7fbdcc102ef0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.259+0000 7fbdd17b2640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 0x7fbdb80a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc8005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.263+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdc409bd30 con 0x7fbdcc102ef0 2024-09-15T23:58:03.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.482+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdcc1a72d0 con 0x7fbdcc102ef0 2024-09-15T23:58:03.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.483+0000 7fbdc27fc640 1 -- 172.21.15.31:0/112543465 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10512 (secure 0 0 0) 0x7fbdc40a0030 con 0x7fbdcc102ef0 2024-09-15T23:58:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:03.485 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"btime":"2024-09-15T23:58:01:746686+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-15T23:58:03.485 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-09-15T23:58:03.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 msgr2=0x7fbdb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:03.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 0x7fbdb80a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c00 tx=0x7fbdc8005e80 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 msgr2=0x7fbdcc1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fd0 tx=0x7fbdc400b260 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 shutdown_connections 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbdb80a3240 0x7fbdb80a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdcc102ef0 0x7fbdcc1041d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdcc102570 0x7fbdcc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 --2- 172.21.15.31:0/112543465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdcc100e20 0x7fbdcc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.485+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 >> 172.21.15.31:0/112543465 conn(0x7fbdcc0fc7d0 msgr2=0x7fbdcc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.486+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 shutdown_connections 2024-09-15T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:03.486+0000 7fbdd3a16640 1 -- 172.21.15.31:0/112543465 wait complete. 2024-09-15T23:58:04.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:04 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.127 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/112543465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:04.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-15T23:58:04.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.389 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.389 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.389 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:04 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.390 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/112543465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:04.425 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:04.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:04 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:04.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/112543465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:04.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 -- 172.21.15.31:0/3302926929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844101420 msgr2=0x7fc84410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:04.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 --2- 172.21.15.31:0/3302926929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844101420 0x7fc84410d950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc834009cd0 tx=0x7fc83402a8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 -- 172.21.15.31:0/3302926929 shutdown_connections 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 --2- 172.21.15.31:0/3302926929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844101420 0x7fc84410d950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 --2- 172.21.15.31:0/3302926929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844100a80 0x7fc844100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 --2- 172.21.15.31:0/3302926929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844106a80 0x7fc844106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 -- 172.21.15.31:0/3302926929 >> 172.21.15.31:0/3302926929 conn(0x7fc8440fc7a0 msgr2=0x7fc8440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 -- 172.21.15.31:0/3302926929 shutdown_connections 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 -- 172.21.15.31:0/3302926929 wait complete. 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.913+0000 7fc849811640 1 Processor -- start 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 -- start start 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844101420 0x7fc844197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844106a80 0x7fc84419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc844110d50 con 0x7fc844100a80 2024-09-15T23:58:04.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc844110bd0 con 0x7fc844106a80 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc849811640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc844110ed0 con 0x7fc844101420 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57116/0 (socket says 172.21.15.31:57116) 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 -- 172.21.15.31:0/1707726513 learned_addr learned my addr 172.21.15.31:0/1707726513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc8437fe640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844106a80 0x7fc84419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc8427fc640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844101420 0x7fc844197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 -- 172.21.15.31:0/1707726513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844101420 msgr2=0x7fc844197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844101420 0x7fc844197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 -- 172.21.15.31:0/1707726513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844106a80 msgr2=0x7fc84419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844106a80 0x7fc84419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.914+0000 7fc842ffd640 1 -- 172.21.15.31:0/1707726513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84419c280 con 0x7fc844100a80 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc842ffd640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b4a0 tx=0x7fc83c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:04.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c014070 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc83c010ac0 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84419c570 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c017c30 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.915+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8441a3e50 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.916+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc844100ee0 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.917+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc83c017070 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.918+0000 7fc84880f640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.918+0000 7fc8427fc640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.918+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc83c0cecf0 con 0x7fc844100a80 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.919+0000 7fc8427fc640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 0x7fc8300a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8440fee50 tx=0x7fc82c005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:04.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:04.921+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc83c09be10 con 0x7fc844100a80 2024-09-15T23:58:05.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.116+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc844198630 con 0x7fc844100a80 2024-09-15T23:58:05.117 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:05 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:05.118 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-09-15T23:58:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-15T23:58:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-09-15T23:58:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-09-15T23:58:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-15T23:58:05.119 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-09-15T23:58:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.116+0000 7fc84880f640 1 -- 172.21.15.31:0/1707726513 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fc83c09b710 con 0x7fc844100a80 2024-09-15T23:58:05.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 msgr2=0x7fc8300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 0x7fc8300a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8440fee50 tx=0x7fc82c005c60 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 msgr2=0x7fc8441972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b4a0 tx=0x7fc83c00b960 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 shutdown_connections 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc8300a3240 0x7fc8300a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc844106a80 0x7fc84419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc844101420 0x7fc844197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 --2- 172.21.15.31:0/1707726513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc844100a80 0x7fc8441972d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.118+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 >> 172.21.15.31:0/1707726513 conn(0x7fc8440fc7a0 msgr2=0x7fc8440fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.119+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 shutdown_connections 2024-09-15T23:58:05.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:05.119+0000 7fc849811640 1 -- 172.21.15.31:0/1707726513 wait complete. 2024-09-15T23:58:05.266 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.267 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:05 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:05 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:58:06.181 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:06 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1707726513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T23:58:06.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:06 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:06 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1707726513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:06 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.542 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T23:58:06.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T23:58:06.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:06 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1707726513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T23:58:06.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:06 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:07.211 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 -- 172.21.15.31:0/883640392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 msgr2=0x7f814010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 --2- 172.21.15.31:0/883640392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f814010d7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8138009cd0 tx=0x7f813802a8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 -- 172.21.15.31:0/883640392 shutdown_connections 2024-09-15T23:58:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 --2- 172.21.15.31:0/883640392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f814010d7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 --2- 172.21.15.31:0/883640392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81401008d0 0x7f8140100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 --2- 172.21.15.31:0/883640392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81401068d0 0x7f8140106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 -- 172.21.15.31:0/883640392 >> 172.21.15.31:0/883640392 conn(0x7f81400fc630 msgr2=0x7f81400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 -- 172.21.15.31:0/883640392 shutdown_connections 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.295+0000 7f8146fa2640 1 -- 172.21.15.31:0/883640392 wait complete. 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 Processor -- start 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 -- start start 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81401008d0 0x7f8140198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81401068d0 0x7f814019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8140110bd0 con 0x7f8140101270 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8140110a50 con 0x7f81401008d0 2024-09-15T23:58:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f8146fa2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8140110d50 con 0x7f81401068d0 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57126/0 (socket says 172.21.15.31:57126) 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 -- 172.21.15.31:0/743244399 learned_addr learned my addr 172.21.15.31:0/743244399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 -- 172.21.15.31:0/743244399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81401068d0 msgr2=0x7f814019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81401068d0 0x7f814019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 -- 172.21.15.31:0/743244399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81401008d0 msgr2=0x7f8140198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81401008d0 0x7f8140198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 -- 172.21.15.31:0/743244399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814019d110 con 0x7f8140101270 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.296+0000 7f814579f640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f813000c890 tx=0x7f813000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.297+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130004220 con 0x7f8140101270 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.297+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814019d400 con 0x7f8140101270 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.297+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81300043c0 con 0x7f8140101270 2024-09-15T23:58:07.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.297+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130004a00 con 0x7f8140101270 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.297+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a4ce0 con 0x7f8140101270 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.298+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8140100d30 con 0x7f8140101270 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.299+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8130004ba0 con 0x7f8140101270 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.300+0000 7f8136ffd640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 0x7f812c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.300+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f81300cf280 con 0x7f8140101270 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.300+0000 7f8145fa0640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 0x7f812c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.300+0000 7f8145fa0640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 0x7f812c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f813c00a860 tx=0x7f813c005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:07.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.304+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8130014030 con 0x7f8140101270 2024-09-15T23:58:07.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:07.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:07.478+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f81401a50b0 con 0x7f8140101270 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:58:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:58:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.107+0000 7f8136ffd640 1 -- 172.21.15.31:0/743244399 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f813009c3a0 con 0x7f8140101270 2024-09-15T23:58:08.108 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-09-15T23:58:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 msgr2=0x7f812c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 0x7f812c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f813c00a860 tx=0x7f813c005b60 comp rx=0 tx=0).stop 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 msgr2=0x7f81401986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f813000c890 tx=0x7f813000cd50 comp rx=0 tx=0).stop 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 shutdown_connections 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f812c0a3240 0x7f812c0a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81401068d0 0x7f814019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8140101270 0x7f81401986a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 --2- 172.21.15.31:0/743244399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81401008d0 0x7f8140198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 >> 172.21.15.31:0/743244399 conn(0x7f81400fc630 msgr2=0x7f814006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.110+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 shutdown_connections 2024-09-15T23:58:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:08.111+0000 7f8146fa2640 1 -- 172.21.15.31:0/743244399 wait complete. 2024-09-15T23:58:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:08 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T23:58:08.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:08 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T23:58:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:08 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T23:58:08.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T23:58:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:09 smithi102 ceph-mon[34458]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:09 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T23:58:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:09 smithi102 ceph-mon[34458]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T23:58:09.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:09 smithi031 ceph-mon[31203]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:09.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:09 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T23:58:09.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:09 smithi031 ceph-mon[31203]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.491+0000 7f610d45e640 1 -- 172.21.15.31:0/3294295591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 msgr2=0x7f610810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.491+0000 7f610d45e640 1 --2- 172.21.15.31:0/3294295591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610810eaa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6100009900 tx=0x7f610002a410 comp rx=0 tx=0).stop 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 -- 172.21.15.31:0/3294295591 shutdown_connections 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 --2- 172.21.15.31:0/3294295591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610810eaa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 --2- 172.21.15.31:0/3294295591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61081027a0 0x7f61081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 --2- 172.21.15.31:0/3294295591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6108101df0 0x7f61081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 -- 172.21.15.31:0/3294295591 >> 172.21.15.31:0/3294295591 conn(0x7f61080fd8b0 msgr2=0x7f61080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 -- 172.21.15.31:0/3294295591 shutdown_connections 2024-09-15T23:58:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 -- 172.21.15.31:0/3294295591 wait complete. 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.492+0000 7f610d45e640 1 Processor -- start 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 -- start start 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6108101df0 0x7f6108198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61081027a0 0x7f6108198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6108111e30 con 0x7f6108106f20 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6108111cb0 con 0x7f6108101df0 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f610d45e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6108111fb0 con 0x7f61081027a0 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57144/0 (socket says 172.21.15.31:57144) 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 -- 172.21.15.31:0/234403371 learned_addr learned my addr 172.21.15.31:0/234403371 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f6106ffd640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6108101df0 0x7f6108198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 -- 172.21.15.31:0/234403371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61081027a0 msgr2=0x7f6108198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61081027a0 0x7f6108198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 -- 172.21.15.31:0/234403371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6108101df0 msgr2=0x7f6108198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6108101df0 0x7f6108198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 -- 172.21.15.31:0/234403371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610819d2e0 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.493+0000 7f61077fe640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6100009230 tx=0x7f6100004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.494+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610003f030 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.494+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610819d570 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.494+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6100003aa0 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.494+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100035070 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.494+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081a4eb0 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.495+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61081021d0 con 0x7f6108106f20 2024-09-15T23:58:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.496+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6100042030 con 0x7f6108106f20 2024-09-15T23:58:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.497+0000 7f60effff640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 0x7f60f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.497+0000 7f6106ffd640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 0x7f60f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.497+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7f61000efd20 con 0x7f6108106f20 2024-09-15T23:58:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.497+0000 7f6106ffd640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 0x7f60f00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005fd0 tx=0x7f60f8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.501+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61000bcd70 con 0x7f6108106f20 2024-09-15T23:58:09.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:09 smithi081 ceph-mon[35060]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:09.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:09 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/743244399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T23:58:09.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:09 smithi081 ceph-mon[35060]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T23:58:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:09.677+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f610810a160 con 0x7f6108106f20 2024-09-15T23:58:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.121+0000 7f60effff640 1 -- 172.21.15.31:0/234403371 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f61000bc670 con 0x7f6108106f20 2024-09-15T23:58:10.122 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 msgr2=0x7f60f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 0x7f60f00a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005fd0 tx=0x7f60f8009680 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 msgr2=0x7f610819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6100009230 tx=0x7f6100004270 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 shutdown_connections 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f60f00a3240 0x7f60f00a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108106f20 0x7f610819cc00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61081027a0 0x7f6108198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 --2- 172.21.15.31:0/234403371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6108101df0 0x7f6108198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:10.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 >> 172.21.15.31:0/234403371 conn(0x7f61080fd8b0 msgr2=0x7f61080febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 shutdown_connections 2024-09-15T23:58:10.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:10.124+0000 7f610d45e640 1 -- 172.21.15.31:0/234403371 wait complete. 2024-09-15T23:58:10.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:10 smithi031 ceph-mon[31203]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T23:58:10.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:10 smithi081 ceph-mon[35060]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T23:58:10.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:10 smithi102 ceph-mon[34458]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T23:58:10.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T23:58:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:11.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:11 smithi031 ceph-mon[31203]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:11.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:11 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T23:58:11.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:11 smithi031 ceph-mon[31203]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T23:58:11.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:11 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T23:58:11.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:11 smithi031 ceph-mon[31203]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T23:58:11.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:11 smithi081 ceph-mon[35060]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:11 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T23:58:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:11 smithi081 ceph-mon[35060]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T23:58:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:11 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T23:58:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:11 smithi081 ceph-mon[35060]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T23:58:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/3544596541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df410bff0 msgr2=0x7f8df410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/3544596541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df410bff0 0x7f8df410e460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009ca0 tx=0x7f8dec02a950 comp rx=0 tx=0).stop 2024-09-15T23:58:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/3544596541 shutdown_connections 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/3544596541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df410bff0 0x7f8df410e460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/3544596541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4103570 0x7f8df410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/3544596541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df4102c50 0x7f8df4103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/3544596541 >> 172.21.15.31:0/3544596541 conn(0x7f8df40fc610 msgr2=0x7f8df40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/3544596541 shutdown_connections 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/3544596541 wait complete. 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 Processor -- start 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.546+0000 7f8dfa5a0640 1 -- start start 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4102c50 0x7f8df4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df410bff0 0x7f8df419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4111c70 con 0x7f8df4103570 2024-09-15T23:58:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4111af0 con 0x7f8df4102c50 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4111df0 con 0x7f8df410bff0 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57166/0 (socket says 172.21.15.31:57166) 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 -- 172.21.15.31:0/783885257 learned_addr learned my addr 172.21.15.31:0/783885257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df9d9f640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df410bff0 0x7f8df419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df959e640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4102c50 0x7f8df4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 -- 172.21.15.31:0/783885257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df410bff0 msgr2=0x7f8df419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df410bff0 0x7f8df419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 -- 172.21.15.31:0/783885257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4102c50 msgr2=0x7f8df4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4102c50 0x7f8df4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 -- 172.21.15.31:0/783885257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df419d120 con 0x7f8df4103570 2024-09-15T23:58:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8df8d9d640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c890 tx=0x7f8de400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4004220 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df419d410 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de40043c0 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4004a00 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.547+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41a4cf0 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.548+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df4103030 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.550+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8de4004ba0 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.551+0000 7f8dea7fc640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 0x7f8de00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.551+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f8de40cf740 con 0x7f8df4103570 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.551+0000 7f8df959e640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 0x7f8de00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.552+0000 7f8df959e640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 0x7f8de00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009bd0 tx=0x7f8df0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.554+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8de4014030 con 0x7f8df4103570 2024-09-15T23:58:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:11 smithi102 ceph-mon[34458]: pgmap v276: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:11 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T23:58:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:11 smithi102 ceph-mon[34458]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T23:58:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:11 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/234403371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T23:58:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:11 smithi102 ceph-mon[34458]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T23:58:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:11.768+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8df41994d0 con 0x7f8df4103570 2024-09-15T23:58:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.138+0000 7f8dea7fc640 1 -- 172.21.15.31:0/783885257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8df41994d0 con 0x7f8df4103570 2024-09-15T23:58:12.140 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T23:58:12.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 msgr2=0x7f8de00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:12.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 0x7f8de00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009bd0 tx=0x7f8df0005f90 comp rx=0 tx=0).stop 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 msgr2=0x7f8df41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c890 tx=0x7f8de400cd50 comp rx=0 tx=0).stop 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 shutdown_connections 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f8de00a3240 0x7f8de00a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8df410bff0 0x7f8df419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:12.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df4103570 0x7f8df41986b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 --2- 172.21.15.31:0/783885257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df4102c50 0x7f8df4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 >> 172.21.15.31:0/783885257 conn(0x7f8df40fc610 msgr2=0x7f8df40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 shutdown_connections 2024-09-15T23:58:12.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:12.141+0000 7f8dfa5a0640 1 -- 172.21.15.31:0/783885257 wait complete. 2024-09-15T23:58:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:12 smithi031 ceph-mon[31203]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T23:58:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:12 smithi031 ceph-mon[31203]: pgmap v279: 97 pgs: 5 creating+peering, 83 unknown, 9 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:12 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T23:58:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:12 smithi031 ceph-mon[31203]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T23:58:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:12 smithi081 ceph-mon[35060]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T23:58:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:12 smithi081 ceph-mon[35060]: pgmap v279: 97 pgs: 5 creating+peering, 83 unknown, 9 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:12 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T23:58:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:12 smithi081 ceph-mon[35060]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T23:58:12.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:12 smithi102 ceph-mon[34458]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T23:58:12.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:12 smithi102 ceph-mon[34458]: pgmap v279: 97 pgs: 5 creating+peering, 83 unknown, 9 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:58:12.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:12 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T23:58:12.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:12 smithi102 ceph-mon[34458]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T23:58:12.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:58:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:13.427 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:13 smithi031 ceph-mon[31203]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T23:58:13.427 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T23:58:13.427 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:13 smithi031 ceph-mon[31203]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T23:58:13.427 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:13 smithi031 ceph-mon[31203]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T23:58:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:13 smithi081 ceph-mon[35060]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T23:58:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T23:58:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:13 smithi081 ceph-mon[35060]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T23:58:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:13 smithi081 ceph-mon[35060]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T23:58:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:13 smithi102 ceph-mon[34458]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T23:58:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/783885257' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T23:58:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:13 smithi102 ceph-mon[34458]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T23:58:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:13 smithi102 ceph-mon[34458]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T23:58:13.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 -- 172.21.15.31:0/2831950547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464106f10 msgr2=0x7f046410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:13.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 --2- 172.21.15.31:0/2831950547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464106f10 0x7f046410ea90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f045c009ca0 tx=0x7f045c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 -- 172.21.15.31:0/2831950547 shutdown_connections 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 --2- 172.21.15.31:0/2831950547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464106f10 0x7f046410ea90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 --2- 172.21.15.31:0/2831950547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464102790 0x7f04641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 --2- 172.21.15.31:0/2831950547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464101de0 0x7f04641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 -- 172.21.15.31:0/2831950547 >> 172.21.15.31:0/2831950547 conn(0x7f04640fd8c0 msgr2=0x7f04640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 -- 172.21.15.31:0/2831950547 shutdown_connections 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.675+0000 7f0469448640 1 -- 172.21.15.31:0/2831950547 wait complete. 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.676+0000 7f0469448640 1 Processor -- start 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.676+0000 7f0469448640 1 -- start start 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.676+0000 7f0469448640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0469448640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464102790 0x7f0464198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0469448640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464106f10 0x7f046419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0469448640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111e20 con 0x7f0464101de0 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0469448640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111ca0 con 0x7f0464106f10 2024-09-15T23:58:13.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0469448640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111fa0 con 0x7f0464102790 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40382/0 (socket says 172.21.15.31:40382) 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 -- 172.21.15.31:0/2137014546 learned_addr learned my addr 172.21.15.31:0/2137014546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f045a7fc640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464102790 0x7f0464198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f04637fe640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464106f10 0x7f046419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 -- 172.21.15.31:0/2137014546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464102790 msgr2=0x7f0464198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464102790 0x7f0464198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 -- 172.21.15.31:0/2137014546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464106f10 msgr2=0x7f046419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464106f10 0x7f046419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 -- 172.21.15.31:0/2137014546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046419d2f0 con 0x7f0464101de0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0462ffd640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f045400d770 tx=0x7f045400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.677+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454017070 con 0x7f0464101de0 2024-09-15T23:58:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.678+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046419d5e0 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.678+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04540054d0 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.678+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004cf0 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.678+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641a4ec0 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.679+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04641021c0 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.680+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0454004e90 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.681+0000 7f0460ff9640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 0x7f04500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.681+0000 7f045a7fc640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 0x7f04500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.681+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f04540cf600 con 0x7f0464101de0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.681+0000 7f045a7fc640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 0x7f04500a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04640fdce0 tx=0x7f044c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.684+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f045409bd80 con 0x7f0464101de0 2024-09-15T23:58:13.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.863+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04641996a0 con 0x7f0464101de0 2024-09-15T23:58:13.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.864+0000 7f0460ff9640 1 -- 172.21.15.31:0/2137014546 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7f04641996a0 con 0x7f0464101de0 2024-09-15T23:58:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:13.865 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.866+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 msgr2=0x7f04500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.866+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 0x7f04500a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04640fdce0 tx=0x7f044c00b390 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.866+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 msgr2=0x7f0464198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.866+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f045400d770 tx=0x7f045400dc30 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 shutdown_connections 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f04500a3240 0x7f04500a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0464106f10 0x7f046419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0464102790 0x7f0464198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 --2- 172.21.15.31:0/2137014546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0464101de0 0x7f0464198340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 >> 172.21.15.31:0/2137014546 conn(0x7f04640fd8c0 msgr2=0x7f04640fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 shutdown_connections 2024-09-15T23:58:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:13.867+0000 7f0469448640 1 -- 172.21.15.31:0/2137014546 wait complete. 2024-09-15T23:58:14.440 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:14 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:active 2024-09-15T23:58:14.440 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:14 smithi031 ceph-mon[31203]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.440 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:14 smithi031 ceph-mon[31203]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:14 smithi031 ceph-mon[31203]: pgmap v281: 97 pgs: 5 creating+peering, 26 unknown, 66 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-15T23:58:14.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2137014546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:14 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:active 2024-09-15T23:58:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:14 smithi081 ceph-mon[35060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:14 smithi081 ceph-mon[35060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:14 smithi081 ceph-mon[35060]: pgmap v281: 97 pgs: 5 creating+peering, 26 unknown, 66 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-15T23:58:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2137014546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:14.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:14 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] up:active 2024-09-15T23:58:14.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:14 smithi102 ceph-mon[34458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:14 smithi102 ceph-mon[34458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:14.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:14 smithi102 ceph-mon[34458]: pgmap v281: 97 pgs: 5 creating+peering, 26 unknown, 66 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-15T23:58:14.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2137014546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:14.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:58:15.039 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:15.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:15 smithi031 ceph-mon[31203]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T23:58:15.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:15 smithi031 ceph-mon[31203]: Cluster is now healthy 2024-09-15T23:58:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 -- 172.21.15.31:0/3524473790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244100e20 msgr2=0x7f4244101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 --2- 172.21.15.31:0/3524473790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244100e20 0x7f4244101200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f423c009ca0 tx=0x7f423c02a950 comp rx=0 tx=0).stop 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 -- 172.21.15.31:0/3524473790 shutdown_connections 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 --2- 172.21.15.31:0/3524473790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102ef0 0x7f424410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 --2- 172.21.15.31:0/3524473790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244102570 0x7f42441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 --2- 172.21.15.31:0/3524473790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244100e20 0x7f4244101200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 -- 172.21.15.31:0/3524473790 >> 172.21.15.31:0/3524473790 conn(0x7f42440fc7d0 msgr2=0x7f42440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 -- 172.21.15.31:0/3524473790 shutdown_connections 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.512+0000 7f424a2f6640 1 -- 172.21.15.31:0/3524473790 wait complete. 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 Processor -- start 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 -- start start 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 0x7f424419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102570 0x7f424419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4244110e30 con 0x7f4244102ef0 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4244110cb0 con 0x7f4244102570 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f424a2f6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4244110fb0 con 0x7f4244100e20 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f4248893640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f4248893640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40400/0 (socket says 172.21.15.31:40400) 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f4248893640 1 -- 172.21.15.31:0/2993772404 learned_addr learned my addr 172.21.15.31:0/2993772404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.513+0000 7f42437fe640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102570 0x7f424419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4243fff640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 0x7f424419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 -- 172.21.15.31:0/2993772404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 msgr2=0x7f424419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 0x7f424419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 -- 172.21.15.31:0/2993772404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102570 msgr2=0x7f424419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102570 0x7f424419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 -- 172.21.15.31:0/2993772404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424419f5a0 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4243fff640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 0x7f424419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f4248893640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f423400d950 tx=0x7f423400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234017030 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4234004020 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424419f890 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234003b20 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.514+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a7170 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.515+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244101200 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.517+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f423401a070 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.517+0000 7f42417fa640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 0x7f422c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.517+0000 7f4243fff640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 0x7f422c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.517+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f42340cf570 con 0x7f4244102ef0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.518+0000 7f4243fff640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 0x7f422c0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f423c009a60 tx=0x7f423c005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.525+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f423409c440 con 0x7f4244102ef0 2024-09-15T23:58:15.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:15 smithi081 ceph-mon[35060]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T23:58:15.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:15 smithi081 ceph-mon[35060]: Cluster is now healthy 2024-09-15T23:58:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:15 smithi102 ceph-mon[34458]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T23:58:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:15 smithi102 ceph-mon[34458]: Cluster is now healthy 2024-09-15T23:58:15.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.706+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f424419b950 con 0x7f4244102ef0 2024-09-15T23:58:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.708+0000 7f42417fa640 1 -- 172.21.15.31:0/2993772404 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7f424419b950 con 0x7f4244102ef0 2024-09-15T23:58:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:15.710 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 msgr2=0x7f422c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 0x7f422c0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f423c009a60 tx=0x7f423c005c30 comp rx=0 tx=0).stop 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 msgr2=0x7f424419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f423400d950 tx=0x7f423400de10 comp rx=0 tx=0).stop 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 shutdown_connections 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f422c0a32c0 0x7f422c0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4244102ef0 0x7f424419eec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4244102570 0x7f424419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 --2- 172.21.15.31:0/2993772404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4244100e20 0x7f424419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:15.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 >> 172.21.15.31:0/2993772404 conn(0x7f42440fc7d0 msgr2=0x7f4244069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:15.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 shutdown_connections 2024-09-15T23:58:15.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:15.711+0000 7f424a2f6640 1 -- 172.21.15.31:0/2993772404 wait complete. 2024-09-15T23:58:16.366 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:16 smithi031 ceph-mon[31203]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-15T23:58:16.367 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:16 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2993772404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:16.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-15T23:58:16.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:16 smithi081 ceph-mon[35060]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-15T23:58:16.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:16 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2993772404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:16.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:16 smithi102 ceph-mon[34458]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-15T23:58:16.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:16 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2993772404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:17.057 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-15T23:58:16.759+0000 7f82c4631640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T23:58:17.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/437250027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 msgr2=0x7f9ec4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:17.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/437250027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec4100f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009900 tx=0x7f9ec002a470 comp rx=0 tx=0).stop 2024-09-15T23:58:17.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/437250027 shutdown_connections 2024-09-15T23:58:17.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/437250027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4105f40 0x7f9ec41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/437250027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4101450 0x7f9ec4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/437250027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec4100f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/437250027 >> 172.21.15.31:0/437250027 conn(0x7f9ec40fc7a0 msgr2=0x7f9ec40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/437250027 shutdown_connections 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.157+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/437250027 wait complete. 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 Processor -- start 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 -- start start 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 0x7f9ec41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4105f40 0x7f9ec419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4118420 con 0x7f9ec4100b30 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41182a0 con 0x7f9ec4101450 2024-09-15T23:58:17.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ecbdc0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41185a0 con 0x7f9ec4105f40 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ec9b5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ec9b5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40410/0 (socket says 172.21.15.31:40410) 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ec9b5c640 1 -- 172.21.15.31:0/2715076491 learned_addr learned my addr 172.21.15.31:0/2715076491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9eca35d640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4105f40 0x7f9ec419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.158+0000 7f9ec935b640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 0x7f9ec41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 -- 172.21.15.31:0/2715076491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4105f40 msgr2=0x7f9ec419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4105f40 0x7f9ec419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 -- 172.21.15.31:0/2715076491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 msgr2=0x7f9ec41a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 0x7f9ec41a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 -- 172.21.15.31:0/2715076491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec419efd0 con 0x7f9ec4100b30 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec935b640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 0x7f9ec41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ec9b5c640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005940 tx=0x7f9ec0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:17.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec003f030 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec419f260 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ec0003aa0 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0035070 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.159+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41ab510 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.160+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec406b070 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.161+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9ec0042030 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.162+0000 7f9ebaffd640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 0x7f9eb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.162+0000 7f9ec935b640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 0x7f9eb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.162+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9ec00eff70 con 0x7f9ec4100b30 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.162+0000 7f9ec935b640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 0x7f9eb00a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005fd0 tx=0x7f9eb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:17.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.166+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ec00bcde0 con 0x7f9ec4100b30 2024-09-15T23:58:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:17.380+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9ec41ab6e0 con 0x7f9ec4100b30 2024-09-15T23:58:18.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.249+0000 7f9ebaffd640 1 -- 172.21.15.31:0/2715076491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9ec00bc6e0 con 0x7f9ec4100b30 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 msgr2=0x7f9eb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 0x7f9eb00a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005fd0 tx=0x7f9eb4005850 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 msgr2=0x7f9ec41a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005940 tx=0x7f9ec0004270 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 shutdown_connections 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9eb00a3240 0x7f9eb00a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ec4105f40 0x7f9ec419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ec4101450 0x7f9ec41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 --2- 172.21.15.31:0/2715076491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ec4100b30 0x7f9ec41a46b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 >> 172.21.15.31:0/2715076491 conn(0x7f9ec40fc7a0 msgr2=0x7f9ec410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 shutdown_connections 2024-09-15T23:58:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:18.252+0000 7f9ecbdc0640 1 -- 172.21.15.31:0/2715076491 wait complete. 2024-09-15T23:58:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:18 smithi031 ceph-mon[31203]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T23:58:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:18 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T23:58:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:18 smithi081 ceph-mon[35060]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T23:58:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:18 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T23:58:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:18 smithi102 ceph-mon[34458]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T23:58:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:18 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T23:58:18.902 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-15T23:58:18.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-15T23:58:19.171 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:19 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T23:58:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:19 smithi081 ceph-mon[35060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:19.561 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:19 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T23:58:19.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:19 smithi031 ceph-mon[31203]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:19.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:19 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2715076491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T23:58:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:19 smithi102 ceph-mon[34458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.645+0000 7f52078fc640 1 -- 172.21.15.31:0/1673807955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200100a80 msgr2=0x7f5200100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.645+0000 7f52078fc640 1 --2- 172.21.15.31:0/1673807955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200100a80 0x7f5200100ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009930 tx=0x7f51f002a420 comp rx=0 tx=0).stop 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- 172.21.15.31:0/1673807955 shutdown_connections 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- 172.21.15.31:0/1673807955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200101420 0x7f520010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- 172.21.15.31:0/1673807955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200100a80 0x7f5200100ee0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- 172.21.15.31:0/1673807955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 0x7f5200106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- 172.21.15.31:0/1673807955 >> 172.21.15.31:0/1673807955 conn(0x7f52000fc7a0 msgr2=0x7f52000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:19.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- 172.21.15.31:0/1673807955 shutdown_connections 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- 172.21.15.31:0/1673807955 wait complete. 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 Processor -- start 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- start start 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200100a80 0x7f52001983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 0x7f520019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110cf0 con 0x7f5200101420 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110b70 con 0x7f5200100a80 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.646+0000 7f52078fc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110e70 con 0x7f5200106a80 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40424/0 (socket says 172.21.15.31:40424) 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 -- 172.21.15.31:0/3429000421 learned_addr learned my addr 172.21.15.31:0/3429000421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5205698640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200100a80 0x7f52001983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5205e99640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 0x7f520019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 -- 172.21.15.31:0/3429000421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 msgr2=0x7f520019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 0x7f520019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 -- 172.21.15.31:0/3429000421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200100a80 msgr2=0x7f52001983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200100a80 0x7f52001983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 -- 172.21.15.31:0/3429000421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520019d370 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f5204e97640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004770 tx=0x7f51f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f003f070 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51f003b460 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0035070 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520019d600 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.647+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a4f40 con 0x7f5200101420 2024-09-15T23:58:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.648+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52001021c0 con 0x7f5200101420 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.650+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f51f003b610 con 0x7f5200101420 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.650+0000 7f51f67fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 0x7f51ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.650+0000 7f5205698640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 0x7f51ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.650+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f51f00efdb0 con 0x7f5200101420 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.651+0000 7f5205698640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 0x7f51ec0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005d00 tx=0x7f51fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.654+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51f00bc5d0 con 0x7f5200101420 2024-09-15T23:58:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:19.864+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f520010a050 con 0x7f5200101420 2024-09-15T23:58:20.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.260+0000 7f51f67fc640 1 -- 172.21.15.31:0/3429000421 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f51f00c08d0 con 0x7f5200101420 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 msgr2=0x7f51ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 0x7f51ec0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005d00 tx=0x7f51fc00b410 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 msgr2=0x7f5200198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51f0004770 tx=0x7f51f00042f0 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 shutdown_connections 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f51ec0a3240 0x7f51ec0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5200106a80 0x7f520019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200101420 0x7f5200198900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 --2- 172.21.15.31:0/3429000421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5200100a80 0x7f52001983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 >> 172.21.15.31:0/3429000421 conn(0x7f52000fc7a0 msgr2=0x7f520006ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 shutdown_connections 2024-09-15T23:58:20.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:20.263+0000 7f52078fc640 1 -- 172.21.15.31:0/3429000421 wait complete. 2024-09-15T23:58:20.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:20 smithi081 ceph-mon[35060]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T23:58:20.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:20 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T23:58:20.590 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:20 smithi031 ceph-mon[31203]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T23:58:20.590 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:20 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T23:58:20.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:20 smithi102 ceph-mon[34458]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T23:58:20.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:20 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T23:58:20.943 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-15T23:58:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:21.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T23:58:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T23:58:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T23:58:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T23:58:21.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:21 smithi031 ceph-mon[31203]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T23:58:21.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T23:58:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T23:58:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T23:58:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T23:58:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-mon[35060]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T23:58:21.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T23:58:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3429000421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T23:58:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T23:58:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T23:58:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T23:58:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:21 smithi102 ceph-mon[34458]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 -- 172.21.15.31:0/841671037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 msgr2=0x7fe544100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 --2- 172.21.15.31:0/841671037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe544100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe534009930 tx=0x7fe53402a420 comp rx=0 tx=0).stop 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 -- 172.21.15.31:0/841671037 shutdown_connections 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 --2- 172.21.15.31:0/841671037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544101420 0x7fe54410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 --2- 172.21.15.31:0/841671037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe544100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 --2- 172.21.15.31:0/841671037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544106a80 0x7fe544106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 -- 172.21.15.31:0/841671037 >> 172.21.15.31:0/841671037 conn(0x7fe5440fc7a0 msgr2=0x7fe5440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 -- 172.21.15.31:0/841671037 shutdown_connections 2024-09-15T23:58:21.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.722+0000 7fe54c144640 1 -- 172.21.15.31:0/841671037 wait complete. 2024-09-15T23:58:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.723+0000 7fe54c144640 1 Processor -- start 2024-09-15T23:58:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.723+0000 7fe54c144640 1 -- start start 2024-09-15T23:58:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.723+0000 7fe54c144640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 0x7fe5441988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544106a80 0x7fe54419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe544111e90 con 0x7fe544100a80 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe544111d10 con 0x7fe544101420 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe544112010 con 0x7fe544106a80 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60256/0 (socket says 172.21.15.31:60256) 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 -- 172.21.15.31:0/549517297 learned_addr learned my addr 172.21.15.31:0/549517297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54a6e1640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544106a80 0x7fe54419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe5496df640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 0x7fe5441988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 -- 172.21.15.31:0/549517297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544106a80 msgr2=0x7fe54419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544106a80 0x7fe54419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 -- 172.21.15.31:0/549517297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 msgr2=0x7fe5441988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 0x7fe5441988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 -- 172.21.15.31:0/549517297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54419d360 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe5496df640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 0x7fe5441988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe549ee0640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c890 tx=0x7fe54000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540004220 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54419d650 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5400043c0 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540004a00 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.724+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5441a4f30 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.726+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe544100ee0 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.727+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe540004ba0 con 0x7fe544100a80 2024-09-15T23:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.728+0000 7fe53affd640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 0x7fe5300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.728+0000 7fe5496df640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 0x7fe5300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.728+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe5400cf500 con 0x7fe544100a80 2024-09-15T23:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.728+0000 7fe5496df640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 0x7fe5300a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe53402a940 tx=0x7fe534005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.732+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe540014030 con 0x7fe544100a80 2024-09-15T23:58:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:21.945+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe544199710 con 0x7fe544100a80 2024-09-15T23:58:22.056 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-15T23:58:21.759+0000 7f82c4631640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-15T23:58:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.275+0000 7fe53affd640 1 -- 172.21.15.31:0/549517297 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fe54009c370 con 0x7fe544100a80 2024-09-15T23:58:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 msgr2=0x7fe5300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 0x7fe5300a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe53402a940 tx=0x7fe534005fd0 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 msgr2=0x7fe5441983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c890 tx=0x7fe54000cd50 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 shutdown_connections 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.277+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe5300a3240 0x7fe5300a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe544106a80 0x7fe54419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe544101420 0x7fe5441988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 --2- 172.21.15.31:0/549517297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe544100a80 0x7fe5441983b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 >> 172.21.15.31:0/549517297 conn(0x7fe5440fc7a0 msgr2=0x7fe5440fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 shutdown_connections 2024-09-15T23:58:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:22.278+0000 7fe54c144640 1 -- 172.21.15.31:0/549517297 wait complete. 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:active 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T23:58:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:22 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:active 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T23:58:22.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:22 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] up:active 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T23:58:22.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:22 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T23:58:23.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-15T23:58:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:23.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T23:58:23.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:active 2024-09-15T23:58:23.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T23:58:23.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T23:58:23.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-15T23:58:23.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:23 smithi031 ceph-mon[31203]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:active 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-15T23:58:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:23 smithi081 ceph-mon[35060]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T23:58:23.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/549517297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T23:58:23.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] up:active 2024-09-15T23:58:23.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T23:58:23.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T23:58:23.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-15T23:58:23.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:23 smithi102 ceph-mon[34458]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T23:58:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 -- 172.21.15.31:0/3176812647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 msgr2=0x7fde201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 --2- 172.21.15.31:0/3176812647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201067e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde10009900 tx=0x7fde1002a410 comp rx=0 tx=0).stop 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 -- 172.21.15.31:0/3176812647 shutdown_connections 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 --2- 172.21.15.31:0/3176812647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20106f10 0x7fde2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 --2- 172.21.15.31:0/3176812647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201067e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 --2- 172.21.15.31:0/3176812647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20101de0 0x7fde201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 -- 172.21.15.31:0/3176812647 >> 172.21.15.31:0/3176812647 conn(0x7fde200fd8c0 msgr2=0x7fde200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 -- 172.21.15.31:0/3176812647 shutdown_connections 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.755+0000 7fde27f73640 1 -- 172.21.15.31:0/3176812647 wait complete. 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 Processor -- start 2024-09-15T23:58:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 -- start start 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 0x7fde20198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20106f10 0x7fde2019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde20111e20 con 0x7fde20102790 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fde20111ca0 con 0x7fde20106f10 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde27f73640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde20111fa0 con 0x7fde20101de0 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde2550e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde2550e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60274/0 (socket says 172.21.15.31:60274) 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde2550e640 1 -- 172.21.15.31:0/3708107628 learned_addr learned my addr 172.21.15.31:0/3708107628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde26510640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20106f10 0x7fde2019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.756+0000 7fde25d0f640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 0x7fde20198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 -- 172.21.15.31:0/3708107628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 msgr2=0x7fde20198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 0x7fde20198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 -- 172.21.15.31:0/3708107628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20106f10 msgr2=0x7fde2019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20106f10 0x7fde2019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 -- 172.21.15.31:0/3708107628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2019d310 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde25d0f640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 0x7fde20198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde2550e640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde10005940 tx=0x7fde100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1003f030 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde10003cb0 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2019d5a0 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10035070 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.757+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde201a4ee0 con 0x7fde20102790 2024-09-15T23:58:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.758+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde201032c0 con 0x7fde20102790 2024-09-15T23:58:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.759+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fde10042030 con 0x7fde20102790 2024-09-15T23:58:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.760+0000 7fde16ffd640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 0x7fde0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.760+0000 7fde25d0f640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 0x7fde0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.760+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fde100effe0 con 0x7fde20102790 2024-09-15T23:58:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.760+0000 7fde25d0f640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 0x7fde0c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009c20 tx=0x7fde1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:23.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.764+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde100bce50 con 0x7fde20102790 2024-09-15T23:58:23.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:23.978+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fde2010a0f0 con 0x7fde20102790 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:active 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: Cluster is now healthy 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-15T23:58:24.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:active 2024-09-15T23:58:24.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:24.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:24 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:active 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: Cluster is now healthy 2024-09-15T23:58:24.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-15T23:58:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-15T23:58:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-15T23:58:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:active 2024-09-15T23:58:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:24.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:24 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T23:58:24.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-09-15T23:58:24.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] up:active 2024-09-15T23:58:24.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T23:58:24.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T23:58:24.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: Cluster is now healthy 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] up:active 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:24.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:24 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T23:58:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.930+0000 7fde16ffd640 1 -- 172.21.15.31:0/3708107628 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fde100bc750 con 0x7fde20102790 2024-09-15T23:58:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 msgr2=0x7fde0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 0x7fde0c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009c20 tx=0x7fde1c005f90 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 msgr2=0x7fde201988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde10005940 tx=0x7fde100042f0 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 shutdown_connections 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fde0c0a3240 0x7fde0c0a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde20106f10 0x7fde2019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.933+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde20102790 0x7fde201988a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.934+0000 7fde27f73640 1 --2- 172.21.15.31:0/3708107628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde20101de0 0x7fde20198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.934+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 >> 172.21.15.31:0/3708107628 conn(0x7fde200fd8c0 msgr2=0x7fde200fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.934+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 shutdown_connections 2024-09-15T23:58:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:24.934+0000 7fde27f73640 1 -- 172.21.15.31:0/3708107628 wait complete. 2024-09-15T23:58:25.717 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-15T23:58:25.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-15T23:58:25.978 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:26.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:25 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T23:58:26.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:25 smithi102 ceph-mon[34458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:26.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:25 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T23:58:26.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:25 smithi031 ceph-mon[31203]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:26.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:25 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3708107628' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T23:58:26.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:25 smithi081 ceph-mon[35060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T23:58:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1564559847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc101df0 msgr2=0x7f2bdc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1564559847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1021d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009900 tx=0x7f2bd802a410 comp rx=0 tx=0).stop 2024-09-15T23:58:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1564559847 shutdown_connections 2024-09-15T23:58:26.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1564559847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc106f20 0x7f2bdc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1564559847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1564559847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1021d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1564559847 >> 172.21.15.31:0/1564559847 conn(0x7f2bdc0fd8b0 msgr2=0x7f2bdc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1564559847 shutdown_connections 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.505+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1564559847 wait complete. 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 Processor -- start 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 -- start start 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 0x7f2bdc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111ed0 con 0x7f2bdc1027a0 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111d50 con 0x7f2bdc101df0 2024-09-15T23:58:26.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be3e5e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc112050 con 0x7f2bdc106f20 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be13f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be13f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60284/0 (socket says 172.21.15.31:60284) 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be13f9640 1 -- 172.21.15.31:0/1567219978 learned_addr learned my addr 172.21.15.31:0/1567219978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be23fb640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 0x7f2bdc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.506+0000 7f2be1bfa640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 -- 172.21.15.31:0/1567219978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 msgr2=0x7f2bdc19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 0x7f2bdc19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 -- 172.21.15.31:0/1567219978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc101df0 msgr2=0x7f2bdc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 -- 172.21.15.31:0/1567219978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc19d580 con 0x7f2bdc1027a0 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be23fb640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 0x7f2bdc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be13f9640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00b4a0 tx=0x7f2bcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:26.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc014070 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bcc010ac0 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc19d870 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc017c30 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.507+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a5150 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.508+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc102250 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.509+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2bcc017070 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.510+0000 7f2bd2ffd640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 0x7f2bc80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.510+0000 7f2be1bfa640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 0x7f2bc80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.510+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2bcc0ceff0 con 0x7f2bdc1027a0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.511+0000 7f2be1bfa640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 0x7f2bc80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031440 tx=0x7f2bd80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:26.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.513+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bcc09bdc0 con 0x7f2bdc1027a0 2024-09-15T23:58:26.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.711+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2bdc062630 con 0x7f2bc80a32c0 2024-09-15T23:58:26.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.732+0000 7f2bd2ffd640 1 -- 172.21.15.31:0/1567219978 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bdc062630 con 0x7f2bc80a32c0 2024-09-15T23:58:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 msgr2=0x7f2bc80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 0x7f2bc80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031440 tx=0x7f2bd80095f0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 msgr2=0x7f2bdc198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00b4a0 tx=0x7f2bcc00b960 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 shutdown_connections 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2bc80a32c0 0x7f2bc80a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bdc106f20 0x7f2bdc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bdc1027a0 0x7f2bdc198b10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 --2- 172.21.15.31:0/1567219978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bdc101df0 0x7f2bdc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.734+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 >> 172.21.15.31:0/1567219978 conn(0x7f2bdc0fd8b0 msgr2=0x7f2bdc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.735+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 shutdown_connections 2024-09-15T23:58:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:26.735+0000 7f2be3e5e640 1 -- 172.21.15.31:0/1567219978 wait complete. 2024-09-15T23:58:26.939 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:58:26.715+0000 7f205976d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:58:26.939 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:58:26.715+0000 7f205976d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:58:26.939 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:58:26.715+0000 7f205976d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:58:26.939 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:58:26.715+0000 7f205976d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:58:26.939 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-15T23:58:26.716+0000 7f205976d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T23:58:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:26 smithi081 ceph-mon[35060]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-15T23:58:27.057 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-15T23:58:26.760+0000 7f82c4631640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-15T23:58:27.057 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 15 23:58:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-15T23:58:26.760+0000 7f82c4631640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-15T23:58:27.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:26 smithi031 ceph-mon[31203]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-15T23:58:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:26 smithi102 ceph-mon[34458]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-15T23:58:27.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-15T23:58:27.908 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:28.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:27 smithi031 ceph-mon[31203]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:28.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:27 smithi081 ceph-mon[35060]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:28.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:27 smithi102 ceph-mon[34458]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 -- 172.21.15.31:0/2532573136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 msgr2=0x7f12d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 --2- 172.21.15.31:0/2532573136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d81067e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009ca0 tx=0x7f12d002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 -- 172.21.15.31:0/2532573136 shutdown_connections 2024-09-15T23:58:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 --2- 172.21.15.31:0/2532573136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 0x7f12d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 --2- 172.21.15.31:0/2532573136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d81067e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 --2- 172.21.15.31:0/2532573136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 0x7f12d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 -- 172.21.15.31:0/2532573136 >> 172.21.15.31:0/2532573136 conn(0x7f12d80fd880 msgr2=0x7f12d80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 -- 172.21.15.31:0/2532573136 shutdown_connections 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.375+0000 7f12de977640 1 -- 172.21.15.31:0/2532573136 wait complete. 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 Processor -- start 2024-09-15T23:58:28.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 -- start start 2024-09-15T23:58:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 0x7f12d819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:28.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 0x7f12d819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8111e60 con 0x7f12d8102790 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8111ce0 con 0x7f12d8101de0 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12de977640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8111fe0 con 0x7f12d8106f10 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60316/0 (socket says 172.21.15.31:60316) 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.376+0000 7f12d77fe640 1 -- 172.21.15.31:0/1881482951 learned_addr learned my addr 172.21.15.31:0/1881482951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12dcf14640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 0x7f12d819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d7fff640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 0x7f12d819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 -- 172.21.15.31:0/1881482951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 msgr2=0x7f12d819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 0x7f12d819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 -- 172.21.15.31:0/1881482951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 msgr2=0x7f12d819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 0x7f12d819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 -- 172.21.15.31:0/1881482951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d819f580 con 0x7f12d8102790 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d77fe640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12d002ade0 tx=0x7f12d0032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0003cf0 con 0x7f12d8102790 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12d00044f0 con 0x7f12d8102790 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d819f810 con 0x7f12d8102790 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0030bd0 con 0x7f12d8102790 2024-09-15T23:58:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.377+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d81a7150 con 0x7f12d8102790 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.378+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d81021c0 con 0x7f12d8102790 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.380+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f12d0030d70 con 0x7f12d8102790 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.380+0000 7f12d57fa640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 0x7f12c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.380+0000 7f12d7fff640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 0x7f12c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.380+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f12d00f0dc0 con 0x7f12d8102790 2024-09-15T23:58:28.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.381+0000 7f12d7fff640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 0x7f12c00a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009990 tx=0x7f12c8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.383+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12d00bdc40 con 0x7f12d8102790 2024-09-15T23:58:28.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.589+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f12d8075430 con 0x7f12c00a3350 2024-09-15T23:58:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.613+0000 7f12d57fa640 1 -- 172.21.15.31:0/1881482951 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12d8075430 con 0x7f12c00a3350 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 msgr2=0x7f12c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 0x7f12c00a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009990 tx=0x7f12c8009110 comp rx=0 tx=0).stop 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 msgr2=0x7f12d819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12d002ade0 tx=0x7f12d0032ff0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 shutdown_connections 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f12c00a3350 0x7f12c00a5810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12d8106f10 0x7f12d819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12d8102790 0x7f12d819ab10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 --2- 172.21.15.31:0/1881482951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12d8101de0 0x7f12d819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 >> 172.21.15.31:0/1881482951 conn(0x7f12d80fd880 msgr2=0x7f12d80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 shutdown_connections 2024-09-15T23:58:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:28.615+0000 7f12de977640 1 -- 172.21.15.31:0/1881482951 wait complete. 2024-09-15T23:58:29.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:28 smithi031 ceph-mon[31203]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T23:58:29.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:28 smithi031 ceph-mon[31203]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T23:58:29.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:28 smithi081 ceph-mon[35060]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T23:58:29.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:28 smithi081 ceph-mon[35060]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T23:58:29.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-15T23:58:29.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:28 smithi102 ceph-mon[34458]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T23:58:29.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:28 smithi102 ceph-mon[34458]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T23:58:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- 172.21.15.31:0/3605212395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204106f10 msgr2=0x7ff20410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/3605212395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204106f10 0x7ff20410ea90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009ca0 tx=0x7ff1fc02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- 172.21.15.31:0/3605212395 shutdown_connections 2024-09-15T23:58:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/3605212395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204106f10 0x7ff20410ea90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/3605212395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff2041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/3605212395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204101de0 0x7ff2041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- 172.21.15.31:0/3605212395 >> 172.21.15.31:0/3605212395 conn(0x7ff2040fd8c0 msgr2=0x7ff2040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- 172.21.15.31:0/3605212395 shutdown_connections 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- 172.21.15.31:0/3605212395 wait complete. 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 Processor -- start 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.040+0000 7ff20c2eb640 1 -- start start 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff204198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204106f10 0x7ff20419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff204111e40 con 0x7ff204101de0 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff204111cc0 con 0x7ff204102790 2024-09-15T23:58:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff204111fc0 con 0x7ff204106f10 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60332/0 (socket says 172.21.15.31:60332) 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 -- 172.21.15.31:0/4281747075 learned_addr learned my addr 172.21.15.31:0/4281747075 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff209886640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff204198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a888640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204106f10 0x7ff20419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 -- 172.21.15.31:0/4281747075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204106f10 msgr2=0x7ff20419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204106f10 0x7ff20419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 -- 172.21.15.31:0/4281747075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 msgr2=0x7ff204198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff204198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 -- 172.21.15.31:0/4281747075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20419d3b0 con 0x7ff204101de0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff209886640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff204198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20a087640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b4a0 tx=0x7ff20000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200014070 con 0x7ff204101de0 2024-09-15T23:58:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff200010ac0 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20419d6a0 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200017c30 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.041+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041a4f80 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.042+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2041021c0 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.044+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff200017070 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.044+0000 7ff1fb7fe640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 0x7ff1f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.044+0000 7ff209886640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 0x7ff1f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.044+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff2000cf060 con 0x7ff204101de0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.045+0000 7ff209886640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 0x7ff1f00a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2040ffe50 tx=0x7ff1f4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.049+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff20009bde0 con 0x7ff204101de0 2024-09-15T23:58:30.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.251+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff20410a150 con 0x7ff1f00a32d0 2024-09-15T23:58:30.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:29 smithi031 ceph-mon[31203]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.257+0000 7ff1fb7fe640 1 -- 172.21.15.31:0/4281747075 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff20410a150 con 0x7ff1f00a32d0 2024-09-15T23:58:30.258 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/ae8a8e28-9c94-4613-9bd6-01b7bcd78699 2024-09-15T23:58:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 msgr2=0x7ff1f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 0x7ff1f00a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2040ffe50 tx=0x7ff1f4005ec0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 msgr2=0x7ff204198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b4a0 tx=0x7ff20000b960 comp rx=0 tx=0).stop 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 shutdown_connections 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff1f00a32d0 0x7ff1f00a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204106f10 0x7ff20419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff204102790 0x7ff204198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 --2- 172.21.15.31:0/4281747075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff204101de0 0x7ff204198400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 >> 172.21.15.31:0/4281747075 conn(0x7ff2040fd8c0 msgr2=0x7ff2040fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:30.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 shutdown_connections 2024-09-15T23:58:30.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:30.259+0000 7ff20c2eb640 1 -- 172.21.15.31:0/4281747075 wait complete. 2024-09-15T23:58:30.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:29 smithi081 ceph-mon[35060]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:30.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:29 smithi102 ceph-mon[34458]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:30.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-15T23:58:31.189 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:31.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:30 smithi031 ceph-mon[31203]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-15T23:58:31.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:30 smithi081 ceph-mon[35060]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-15T23:58:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:30 smithi102 ceph-mon[34458]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-15T23:58:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 -- 172.21.15.31:0/2900524770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578101420 msgr2=0x7fd57810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/2900524770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578101420 0x7fd57810d950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd570009cd0 tx=0x7fd57002a8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 -- 172.21.15.31:0/2900524770 shutdown_connections 2024-09-15T23:58:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/2900524770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578101420 0x7fd57810d950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/2900524770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd578100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/2900524770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578106a80 0x7fd578106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 -- 172.21.15.31:0/2900524770 >> 172.21.15.31:0/2900524770 conn(0x7fd5780fc7a0 msgr2=0x7fd5780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 -- 172.21.15.31:0/2900524770 shutdown_connections 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.667+0000 7fd57e2ec640 1 -- 172.21.15.31:0/2900524770 wait complete. 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 Processor -- start 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 -- start start 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd5780fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 0x7fd5780fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd578110dd0 con 0x7fd578106a80 2024-09-15T23:58:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd578110c50 con 0x7fd578100a80 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57e2ec640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd578110f50 con 0x7fd578101420 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57c889640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57c889640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59864/0 (socket says 172.21.15.31:59864) 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd57c889640 1 -- 172.21.15.31:0/3896547718 learned_addr learned my addr 172.21.15.31:0/3896547718 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.668+0000 7fd5777fe640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 0x7fd5780fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd577fff640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd5780fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 -- 172.21.15.31:0/3896547718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 msgr2=0x7fd5780fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 0x7fd5780fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 -- 172.21.15.31:0/3896547718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 msgr2=0x7fd5780fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd5780fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 -- 172.21.15.31:0/3896547718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5781a6950 con 0x7fd578106a80 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd5777fe640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 0x7fd5780fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd577fff640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd5780fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57c889640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd570005cc0 tx=0x7fd570003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570004240 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5700305a0 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5781a6be0 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700466c0 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.669+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5781a7120 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.670+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd578100f60 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.671+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd570030ac0 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.672+0000 7fd5757fa640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 0x7fd5600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.672+0000 7fd577fff640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 0x7fd5600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.672+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd57002e030 con 0x7fd578106a80 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.672+0000 7fd577fff640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 0x7fd5600a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd568006fd0 tx=0x7fd568008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.676+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd57004b060 con 0x7fd578106a80 2024-09-15T23:58:31.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.877+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd578062630 con 0x7fd5600a32d0 2024-09-15T23:58:31.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.900+0000 7fd5757fa640 1 -- 172.21.15.31:0/3896547718 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd578062630 con 0x7fd5600a32d0 2024-09-15T23:58:31.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 msgr2=0x7fd5600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 0x7fd5600a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd568006fd0 tx=0x7fd568008040 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 msgr2=0x7fd5780fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd570005cc0 tx=0x7fd570003ba0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 shutdown_connections 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd5600a32d0 0x7fd5600a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd578106a80 0x7fd5780fd940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd578101420 0x7fd5780fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 --2- 172.21.15.31:0/3896547718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd578100a80 0x7fd5780fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 >> 172.21.15.31:0/3896547718 conn(0x7fd5780fc7a0 msgr2=0x7fd57806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 shutdown_connections 2024-09-15T23:58:31.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:31.902+0000 7fd57e2ec640 1 -- 172.21.15.31:0/3896547718 wait complete. 2024-09-15T23:58:32.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:31 smithi031 ceph-mon[31203]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:32.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:31 smithi081 ceph-mon[35060]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:32.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:31 smithi102 ceph-mon[34458]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:32.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-15T23:58:32.898 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:32 smithi031 ceph-mon[31203]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-09-15T23:58:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:32 smithi031 ceph-mon[31203]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:32 smithi081 ceph-mon[35060]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-09-15T23:58:33.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:32 smithi081 ceph-mon[35060]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:32 smithi102 ceph-mon[34458]: pgmap v290: 97 pgs: 97 active+clean; 591 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-09-15T23:58:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:32 smithi102 ceph-mon[34458]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 -- 172.21.15.31:0/2874876189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724106f10 msgr2=0x7f072410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 --2- 172.21.15.31:0/2874876189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724106f10 0x7f072410ea90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f071c009ca0 tx=0x7f071c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 -- 172.21.15.31:0/2874876189 shutdown_connections 2024-09-15T23:58:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 --2- 172.21.15.31:0/2874876189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724106f10 0x7f072410ea90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 --2- 172.21.15.31:0/2874876189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724102790 0x7f07241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 --2- 172.21.15.31:0/2874876189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724101de0 0x7f07241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.376+0000 7f0729cfc640 1 -- 172.21.15.31:0/2874876189 >> 172.21.15.31:0/2874876189 conn(0x7f07240fd8c0 msgr2=0x7f07240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- 172.21.15.31:0/2874876189 shutdown_connections 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- 172.21.15.31:0/2874876189 wait complete. 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 Processor -- start 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- start start 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 0x7f0724198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724106f10 0x7f072419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111e20 con 0x7f0724101de0 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111ca0 con 0x7f0724102790 2024-09-15T23:58:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f0729cfc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111fa0 con 0x7f0724106f10 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f07237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f07237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59878/0 (socket says 172.21.15.31:59878) 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.377+0000 7f07237fe640 1 -- 172.21.15.31:0/3851207567 learned_addr learned my addr 172.21.15.31:0/3851207567 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f0723fff640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724106f10 0x7f072419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f071bfff640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 0x7f0724198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 -- 172.21.15.31:0/3851207567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724106f10 msgr2=0x7f072419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724106f10 0x7f072419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 -- 172.21.15.31:0/3851207567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 msgr2=0x7f0724198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 0x7f0724198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 -- 172.21.15.31:0/3851207567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072419d1f0 con 0x7f0724101de0 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f071bfff640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 0x7f0724198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07237fe640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f071400c920 tx=0x7f071400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714007c70 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0714007e10 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072419d4e0 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a4dc0 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.378+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714018820 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.379+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0704005290 con 0x7f0724101de0 2024-09-15T23:58:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.381+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f07140189c0 con 0x7f0724101de0 2024-09-15T23:58:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.381+0000 7f07217fa640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 0x7f07100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.382+0000 7f071bfff640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 0x7f07100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.382+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f07140cf8d0 con 0x7f0724101de0 2024-09-15T23:58:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.382+0000 7f071bfff640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 0x7f07100a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07241088e0 tx=0x7f070c005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.386+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0714014030 con 0x7f0724101de0 2024-09-15T23:58:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.590+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0704000bc0 con 0x7f07100a3350 2024-09-15T23:58:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.594+0000 7f07217fa640 1 -- 172.21.15.31:0/3851207567 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0704000bc0 con 0x7f07100a3350 2024-09-15T23:58:33.595 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/a6d2b80c-baa0-45c0-86eb-1f98856a9e7c 2024-09-15T23:58:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 msgr2=0x7f07100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 0x7f07100a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07241088e0 tx=0x7f070c005cb0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 msgr2=0x7f0724198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f071400c920 tx=0x7f071400cde0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 shutdown_connections 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f07100a3350 0x7f07100a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0724106f10 0x7f072419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0724102790 0x7f0724198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 --2- 172.21.15.31:0/3851207567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0724101de0 0x7f0724198240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.596+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 >> 172.21.15.31:0/3851207567 conn(0x7f07240fd8c0 msgr2=0x7f07240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.597+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 shutdown_connections 2024-09-15T23:58:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:33.597+0000 7f0729cfc640 1 -- 172.21.15.31:0/3851207567 wait complete. 2024-09-15T23:58:34.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:58:34.692 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:35.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:34 smithi081 ceph-mon[35060]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-09-15T23:58:35.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:34 smithi081 ceph-mon[35060]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:35.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:34 smithi031 ceph-mon[31203]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-09-15T23:58:35.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:34 smithi031 ceph-mon[31203]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:35.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 -- 172.21.15.31:0/3772058122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4102790 msgr2=0x7fd0f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:35.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/3772058122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4102790 0x7fd0f41067e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009900 tx=0x7fd0e802a410 comp rx=0 tx=0).stop 2024-09-15T23:58:35.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 -- 172.21.15.31:0/3772058122 shutdown_connections 2024-09-15T23:58:35.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/3772058122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4106f10 0x7fd0f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/3772058122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4102790 0x7fd0f41067e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/3772058122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 0x7fd0f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 -- 172.21.15.31:0/3772058122 >> 172.21.15.31:0/3772058122 conn(0x7fd0f40fd8c0 msgr2=0x7fd0f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 -- 172.21.15.31:0/3772058122 shutdown_connections 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.166+0000 7fd0fb68f640 1 -- 172.21.15.31:0/3772058122 wait complete. 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 Processor -- start 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 -- start start 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 0x7fd0f4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:35.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4102790 0x7fd0f41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:35.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:35.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4111e20 con 0x7fd0f4106f10 2024-09-15T23:58:35.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4111ca0 con 0x7fd0f4102790 2024-09-15T23:58:35.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0fb68f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4111fa0 con 0x7fd0f4101de0 2024-09-15T23:58:35.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59906/0 (socket says 172.21.15.31:59906) 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 -- 172.21.15.31:0/2110023035 learned_addr learned my addr 172.21.15.31:0/2110023035 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f942b640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 0x7fd0f4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f8c2a640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4102790 0x7fd0f41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 -- 172.21.15.31:0/2110023035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 msgr2=0x7fd0f4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 0x7fd0f4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 -- 172.21.15.31:0/2110023035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4102790 msgr2=0x7fd0f41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4102790 0x7fd0f41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.167+0000 7fd0f9c2c640 1 -- 172.21.15.31:0/2110023035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f419d330 con 0x7fd0f4106f10 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0f9c2c640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00e970 tx=0x7fd0ec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:35.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00ccc0 con 0x7fd0f4106f10 2024-09-15T23:58:35.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0ec00ce60 con 0x7fd0f4106f10 2024-09-15T23:58:35.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f419d620 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec010430 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.168+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f41a4f00 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.169+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f41021c0 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.170+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd0ec004ff0 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.171+0000 7fd0f27fc640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 0x7fd0dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.171+0000 7fd0f942b640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 0x7fd0dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.171+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd0ec0cfc10 con 0x7fd0f4106f10 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.171+0000 7fd0f942b640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 0x7fd0dc0a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005fd0 tx=0x7fd0e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:35.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.175+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0ec09c980 con 0x7fd0f4106f10 2024-09-15T23:58:35.254 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:34 smithi102 ceph-mon[34458]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-09-15T23:58:35.254 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:34 smithi102 ceph-mon[34458]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:58:35.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.387+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0f410a0f0 con 0x7fd0f4106f10 2024-09-15T23:58:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.387+0000 7fd0f27fc640 1 -- 172.21.15.31:0/2110023035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7fd0ec09c280 con 0x7fd0f4106f10 2024-09-15T23:58:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:35.389 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:58:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:58:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 msgr2=0x7fd0dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 0x7fd0dc0a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005fd0 tx=0x7fd0e4009500 comp rx=0 tx=0).stop 2024-09-15T23:58:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 msgr2=0x7fd0f419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00e970 tx=0x7fd0ec00ed70 comp rx=0 tx=0).stop 2024-09-15T23:58:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 shutdown_connections 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd0dc0a3350 0x7fd0dc0a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f4106f10 0x7fd0f419cc50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f4102790 0x7fd0f41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 --2- 172.21.15.31:0/2110023035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0f4101de0 0x7fd0f4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.389+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 >> 172.21.15.31:0/2110023035 conn(0x7fd0f40fd8c0 msgr2=0x7fd0f40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.390+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 shutdown_connections 2024-09-15T23:58:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:35.390+0000 7fd0fb68f640 1 -- 172.21.15.31:0/2110023035 wait complete. 2024-09-15T23:58:36.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:35 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2110023035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:36.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:35 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2110023035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:36.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:58:36.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:35 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2110023035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:58:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 -- 172.21.15.31:0/1655028176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 msgr2=0x7f265810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 --2- 172.21.15.31:0/1655028176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265810eaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2650009ca0 tx=0x7f265002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 -- 172.21.15.31:0/1655028176 shutdown_connections 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 --2- 172.21.15.31:0/1655028176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265810eaa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 --2- 172.21.15.31:0/1655028176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26581027a0 0x7f26581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 --2- 172.21.15.31:0/1655028176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2658101df0 0x7f26581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 -- 172.21.15.31:0/1655028176 >> 172.21.15.31:0/1655028176 conn(0x7f26580fd8f0 msgr2=0x7f26580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:36.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 -- 172.21.15.31:0/1655028176 shutdown_connections 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.973+0000 7f265e57b640 1 -- 172.21.15.31:0/1655028176 wait complete. 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 Processor -- start 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 -- start start 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2658101df0 0x7f26581982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26581027a0 0x7f2658198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111e30 con 0x7f2658106f20 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111cb0 con 0x7f26581027a0 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265e57b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111fb0 con 0x7f2658101df0 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265cb18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:36.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265cb18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59924/0 (socket says 172.21.15.31:59924) 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f265cb18640 1 -- 172.21.15.31:0/3949542430 learned_addr learned my addr 172.21.15.31:0/3949542430 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f264ffff640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26581027a0 0x7f2658198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.974+0000 7f2657fff640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2658101df0 0x7f26581982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 -- 172.21.15.31:0/3949542430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2658101df0 msgr2=0x7f26581982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2658101df0 0x7f26581982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 -- 172.21.15.31:0/3949542430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26581027a0 msgr2=0x7f2658198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26581027a0 0x7f2658198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 -- 172.21.15.31:0/3949542430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265819d2a0 con 0x7f2658106f20 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265cb18640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2650009dc0 tx=0x7f2650032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650003bf0 con 0x7f2658106f20 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265819d530 con 0x7f2658106f20 2024-09-15T23:58:36.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2650003d90 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650030b80 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.975+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a4e70 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.976+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26581021d0 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.977+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2650004040 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.978+0000 7f2655ffb640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 0x7f26440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.978+0000 7f2657fff640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 0x7f26440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.978+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f26500f0f20 con 0x7f2658106f20 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.978+0000 7f2657fff640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 0x7f26440a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2648009bd0 tx=0x7f2648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:36.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:36.982+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26500bdca0 con 0x7f2658106f20 2024-09-15T23:58:37.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.160+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f265810a160 con 0x7f2658106f20 2024-09-15T23:58:37.161 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:36 smithi031 ceph-mon[31203]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-09-15T23:58:37.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.162+0000 7f2655ffb640 1 -- 172.21.15.31:0/3949542430 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7f26500bd5a0 con 0x7f2658106f20 2024-09-15T23:58:37.163 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:37.163 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 msgr2=0x7f26440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 0x7f26440a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2648009bd0 tx=0x7f2648005f90 comp rx=0 tx=0).stop 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 msgr2=0x7f265819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2650009dc0 tx=0x7f2650032ff0 comp rx=0 tx=0).stop 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 shutdown_connections 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f26440a32d0 0x7f26440a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:37.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2658106f20 0x7f265819cbc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:37.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26581027a0 0x7f2658198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:37.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 --2- 172.21.15.31:0/3949542430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2658101df0 0x7f26581982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:37.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 >> 172.21.15.31:0/3949542430 conn(0x7f26580fd8f0 msgr2=0x7f26580fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:37.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 shutdown_connections 2024-09-15T23:58:37.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:37.164+0000 7f265e57b640 1 -- 172.21.15.31:0/3949542430 wait complete. 2024-09-15T23:58:37.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:36 smithi081 ceph-mon[35060]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-09-15T23:58:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:36 smithi102 ceph-mon[34458]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-09-15T23:58:37.990 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T23:58:37.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T23:58:38.254 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:38.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:37 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3949542430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:38.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:37 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3949542430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:38.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:37 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3949542430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:38.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- 172.21.15.31:0/3255545491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100cc0 msgr2=0x7fb054106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 --2- 172.21.15.31:0/3255545491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100cc0 0x7fb054106800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb048009900 tx=0x7fb04802a410 comp rx=0 tx=0).stop 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- 172.21.15.31:0/3255545491 shutdown_connections 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 --2- 172.21.15.31:0/3255545491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 --2- 172.21.15.31:0/3255545491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100cc0 0x7fb054106800 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 --2- 172.21.15.31:0/3255545491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100310 0x7fb0541006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- 172.21.15.31:0/3255545491 >> 172.21.15.31:0/3255545491 conn(0x7fb054077210 msgr2=0x7fb0540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- 172.21.15.31:0/3255545491 shutdown_connections 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- 172.21.15.31:0/3255545491 wait complete. 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 Processor -- start 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.755+0000 7fb05c64d640 1 -- start start 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:38.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100cc0 0x7fb054198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111e90 con 0x7fb054100310 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111d10 con 0x7fb054100cc0 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb054112010 con 0x7fb054106f30 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59950/0 (socket says 172.21.15.31:59950) 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 -- 172.21.15.31:0/817819066 learned_addr learned my addr 172.21.15.31:0/817819066 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb059be8640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100cc0 0x7fb054198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05abea640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 -- 172.21.15.31:0/817819066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 msgr2=0x7fb05419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 -- 172.21.15.31:0/817819066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100cc0 msgr2=0x7fb054198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100cc0 0x7fb054198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 -- 172.21.15.31:0/817819066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05419d380 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05abea640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05a3e9640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb04400eab0 tx=0x7fb04400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04400ce40 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb044011920 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05419d670 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044010a00 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.756+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a4f50 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.757+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0541006f0 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.759+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb044011ad0 con 0x7fb054100310 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.759+0000 7fb0537fe640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 0x7fb03c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.759+0000 7fb059be8640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 0x7fb03c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:38.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.759+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb0440cff40 con 0x7fb054100310 2024-09-15T23:58:38.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.760+0000 7fb059be8640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 0x7fb03c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb048005ab0 tx=0x7fb048005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:38.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.763+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb044016030 con 0x7fb054100310 2024-09-15T23:58:38.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.936+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb054062630 con 0x7fb03c0a32d0 2024-09-15T23:58:38.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.942+0000 7fb0537fe640 1 -- 172.21.15.31:0/817819066 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7fb054062630 con 0x7fb03c0a32d0 2024-09-15T23:58:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 msgr2=0x7fb03c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 0x7fb03c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb048005ab0 tx=0x7fb048005a20 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 msgr2=0x7fb0541983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb04400eab0 tx=0x7fb04400ef70 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 shutdown_connections 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb03c0a32d0 0x7fb03c0a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb054106f30 0x7fb05419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb054100cc0 0x7fb054198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 --2- 172.21.15.31:0/817819066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb054100310 0x7fb0541983d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 >> 172.21.15.31:0/817819066 conn(0x7fb054077210 msgr2=0x7fb054105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 shutdown_connections 2024-09-15T23:58:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:38.945+0000 7fb05c64d640 1 -- 172.21.15.31:0/817819066 wait complete. 2024-09-15T23:58:39.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:38 smithi031 ceph-mon[31203]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-15T23:58:39.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:38 smithi081 ceph-mon[35060]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-15T23:58:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:38 smithi102 ceph-mon[34458]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-09-15T23:58:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-15T23:58:37.192402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638482,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":275,"num_read_kb":241,"num_write":408,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347508,"kb_used_data":10740,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594316,"statfs":{"total":1151940427776,"available":1151584579584,"internally_reserved":0,"allocated":10997760,"data_stored":6909858,"data_compressed":107100,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":344634682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41452,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":121,"num_read_kb":107,"num_write":64,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002865"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154743+0000","last_change":"2024-09-15T23:58:11.160319+0000","last_active":"2024-09-15T23:58:12.154743+0000","last_peered":"2024-09-15T23:58:12.154743+0000","last_clean":"2024-09-15T23:58:12.154743+0000","last_became_active":"2024-09-15T23:58:11.159572+0000","last_became_peered":"2024-09-15T23:58:11.159572+0000","last_unstale":"2024-09-15T23:58:12.154743+0000","last_undegraded":"2024-09-15T23:58:12.154743+0000","last_fullsized":"2024-09-15T23:58:12.154743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:50.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150768+0000","last_change":"2024-09-15T23:58:11.152044+0000","last_active":"2024-09-15T23:58:12.150768+0000","last_peered":"2024-09-15T23:58:12.150768+0000","last_clean":"2024-09-15T23:58:12.150768+0000","last_became_active":"2024-09-15T23:58:11.151472+0000","last_became_peered":"2024-09-15T23:58:11.151472+0000","last_unstale":"2024-09-15T23:58:12.150768+0000","last_undegraded":"2024-09-15T23:58:12.150768+0000","last_fullsized":"2024-09-15T23:58:12.150768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:51.502165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150571+0000","last_change":"2024-09-15T23:58:11.151854+0000","last_active":"2024-09-15T23:58:12.150571+0000","last_peered":"2024-09-15T23:58:12.150571+0000","last_clean":"2024-09-15T23:58:12.150571+0000","last_became_active":"2024-09-15T23:58:11.151418+0000","last_became_peered":"2024-09-15T23:58:11.151418+0000","last_unstale":"2024-09-15T23:58:12.150571+0000","last_undegraded":"2024-09-15T23:58:12.150571+0000","last_fullsized":"2024-09-15T23:58:12.150571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:48.513653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149324+0000","last_change":"2024-09-15T23:58:11.162524+0000","last_active":"2024-09-15T23:58:12.149324+0000","last_peered":"2024-09-15T23:58:12.149324+0000","last_clean":"2024-09-15T23:58:12.149324+0000","last_became_active":"2024-09-15T23:58:11.161994+0000","last_became_peered":"2024-09-15T23:58:11.161994+0000","last_unstale":"2024-09-15T23:58:12.149324+0000","last_undegraded":"2024-09-15T23:58:12.149324+0000","last_fullsized":"2024-09-15T23:58:12.149324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:25.333515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150250+0000","last_change":"2024-09-15T23:58:11.143229+0000","last_active":"2024-09-15T23:58:12.150250+0000","last_peered":"2024-09-15T23:58:12.150250+0000","last_clean":"2024-09-15T23:58:12.150250+0000","last_became_active":"2024-09-15T23:58:11.142346+0000","last_became_peered":"2024-09-15T23:58:11.142346+0000","last_unstale":"2024-09-15T23:58:12.150250+0000","last_undegraded":"2024-09-15T23:58:12.150250+0000","last_fullsized":"2024-09-15T23:58:12.150250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:51.414916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136944+0000","last_change":"2024-09-15T23:58:11.152092+0000","last_active":"2024-09-15T23:58:12.136944+0000","last_peered":"2024-09-15T23:58:12.136944+0000","last_clean":"2024-09-15T23:58:12.136944+0000","last_became_active":"2024-09-15T23:58:11.151769+0000","last_became_peered":"2024-09-15T23:58:11.151769+0000","last_unstale":"2024-09-15T23:58:12.136944+0000","last_undegraded":"2024-09-15T23:58:12.136944+0000","last_fullsized":"2024-09-15T23:58:12.136944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:42.690545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150191+0000","last_change":"2024-09-15T23:58:11.158671+0000","last_active":"2024-09-15T23:58:12.150191+0000","last_peered":"2024-09-15T23:58:12.150191+0000","last_clean":"2024-09-15T23:58:12.150191+0000","last_became_active":"2024-09-15T23:58:11.158347+0000","last_became_peered":"2024-09-15T23:58:11.158347+0000","last_unstale":"2024-09-15T23:58:12.150191+0000","last_undegraded":"2024-09-15T23:58:12.150191+0000","last_fullsized":"2024-09-15T23:58:12.150191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:52.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136298+0000","last_change":"2024-09-15T23:58:11.149725+0000","last_active":"2024-09-15T23:58:12.136298+0000","last_peered":"2024-09-15T23:58:12.136298+0000","last_clean":"2024-09-15T23:58:12.136298+0000","last_became_active":"2024-09-15T23:58:11.149388+0000","last_became_peered":"2024-09-15T23:58:11.149388+0000","last_unstale":"2024-09-15T23:58:12.136298+0000","last_undegraded":"2024-09-15T23:58:12.136298+0000","last_fullsized":"2024-09-15T23:58:12.136298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:54.807443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149128+0000","last_change":"2024-09-15T23:58:11.155976+0000","last_active":"2024-09-15T23:58:12.149128+0000","last_peered":"2024-09-15T23:58:12.149128+0000","last_clean":"2024-09-15T23:58:12.149128+0000","last_became_active":"2024-09-15T23:58:11.155659+0000","last_became_peered":"2024-09-15T23:58:11.155659+0000","last_unstale":"2024-09-15T23:58:12.149128+0000","last_undegraded":"2024-09-15T23:58:12.149128+0000","last_fullsized":"2024-09-15T23:58:12.149128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:21.524908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137172+0000","last_change":"2024-09-15T23:58:11.157326+0000","last_active":"2024-09-15T23:58:12.137172+0000","last_peered":"2024-09-15T23:58:12.137172+0000","last_clean":"2024-09-15T23:58:12.137172+0000","last_became_active":"2024-09-15T23:58:11.156944+0000","last_became_peered":"2024-09-15T23:58:11.156944+0000","last_unstale":"2024-09-15T23:58:12.137172+0000","last_undegraded":"2024-09-15T23:58:12.137172+0000","last_fullsized":"2024-09-15T23:58:12.137172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:49.913437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151176+0000","last_change":"2024-09-15T23:58:11.154418+0000","last_active":"2024-09-15T23:58:12.151176+0000","last_peered":"2024-09-15T23:58:12.151176+0000","last_clean":"2024-09-15T23:58:12.151176+0000","last_became_active":"2024-09-15T23:58:11.154081+0000","last_became_peered":"2024-09-15T23:58:11.154081+0000","last_unstale":"2024-09-15T23:58:12.151176+0000","last_undegraded":"2024-09-15T23:58:12.151176+0000","last_fullsized":"2024-09-15T23:58:12.151176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:09.920736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149482+0000","last_change":"2024-09-15T23:58:11.158224+0000","last_active":"2024-09-15T23:58:12.149482+0000","last_peered":"2024-09-15T23:58:12.149482+0000","last_clean":"2024-09-15T23:58:12.149482+0000","last_became_active":"2024-09-15T23:58:11.156076+0000","last_became_peered":"2024-09-15T23:58:11.156076+0000","last_unstale":"2024-09-15T23:58:12.149482+0000","last_undegraded":"2024-09-15T23:58:12.149482+0000","last_fullsized":"2024-09-15T23:58:12.149482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:29.448669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148503+0000","last_change":"2024-09-15T23:58:11.151904+0000","last_active":"2024-09-15T23:58:12.148503+0000","last_peered":"2024-09-15T23:58:12.148503+0000","last_clean":"2024-09-15T23:58:12.148503+0000","last_became_active":"2024-09-15T23:58:11.151564+0000","last_became_peered":"2024-09-15T23:58:11.151564+0000","last_unstale":"2024-09-15T23:58:12.148503+0000","last_undegraded":"2024-09-15T23:58:12.148503+0000","last_fullsized":"2024-09-15T23:58:12.148503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:11.297350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151021+0000","last_change":"2024-09-15T23:58:11.148139+0000","last_active":"2024-09-15T23:58:12.151021+0000","last_peered":"2024-09-15T23:58:12.151021+0000","last_clean":"2024-09-15T23:58:12.151021+0000","last_became_active":"2024-09-15T23:58:11.147820+0000","last_became_peered":"2024-09-15T23:58:11.147820+0000","last_unstale":"2024-09-15T23:58:12.151021+0000","last_undegraded":"2024-09-15T23:58:12.151021+0000","last_fullsized":"2024-09-15T23:58:12.151021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:33.786315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150504+0000","last_change":"2024-09-15T23:58:11.160284+0000","last_active":"2024-09-15T23:58:12.150504+0000","last_peered":"2024-09-15T23:58:12.150504+0000","last_clean":"2024-09-15T23:58:12.150504+0000","last_became_active":"2024-09-15T23:58:11.159931+0000","last_became_peered":"2024-09-15T23:58:11.159931+0000","last_unstale":"2024-09-15T23:58:12.150504+0000","last_undegraded":"2024-09-15T23:58:12.150504+0000","last_fullsized":"2024-09-15T23:58:12.150504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:35.381030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147613+0000","last_change":"2024-09-15T23:58:11.162637+0000","last_active":"2024-09-15T23:58:12.147613+0000","last_peered":"2024-09-15T23:58:12.147613+0000","last_clean":"2024-09-15T23:58:12.147613+0000","last_became_active":"2024-09-15T23:58:11.162322+0000","last_became_peered":"2024-09-15T23:58:11.162322+0000","last_unstale":"2024-09-15T23:58:12.147613+0000","last_undegraded":"2024-09-15T23:58:12.147613+0000","last_fullsized":"2024-09-15T23:58:12.147613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:50.489012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154813+0000","last_change":"2024-09-15T23:58:11.160068+0000","last_active":"2024-09-15T23:58:12.154813+0000","last_peered":"2024-09-15T23:58:12.154813+0000","last_clean":"2024-09-15T23:58:12.154813+0000","last_became_active":"2024-09-15T23:58:11.159553+0000","last_became_peered":"2024-09-15T23:58:11.159553+0000","last_unstale":"2024-09-15T23:58:12.154813+0000","last_undegraded":"2024-09-15T23:58:12.154813+0000","last_fullsized":"2024-09-15T23:58:12.154813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:31.962049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147516+0000","last_change":"2024-09-15T23:58:11.148755+0000","last_active":"2024-09-15T23:58:12.147516+0000","last_peered":"2024-09-15T23:58:12.147516+0000","last_clean":"2024-09-15T23:58:12.147516+0000","last_became_active":"2024-09-15T23:58:11.148428+0000","last_became_peered":"2024-09-15T23:58:11.148428+0000","last_unstale":"2024-09-15T23:58:12.147516+0000","last_undegraded":"2024-09-15T23:58:12.147516+0000","last_fullsized":"2024-09-15T23:58:12.147516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.119847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148320+0000","last_change":"2024-09-15T23:58:11.143374+0000","last_active":"2024-09-15T23:58:12.148320+0000","last_peered":"2024-09-15T23:58:12.148320+0000","last_clean":"2024-09-15T23:58:12.148320+0000","last_became_active":"2024-09-15T23:58:11.143032+0000","last_became_peered":"2024-09-15T23:58:11.143032+0000","last_unstale":"2024-09-15T23:58:12.148320+0000","last_undegraded":"2024-09-15T23:58:12.148320+0000","last_fullsized":"2024-09-15T23:58:12.148320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:29.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153498+0000","last_change":"2024-09-15T23:58:11.160524+0000","last_active":"2024-09-15T23:58:12.153498+0000","last_peered":"2024-09-15T23:58:12.153498+0000","last_clean":"2024-09-15T23:58:12.153498+0000","last_became_active":"2024-09-15T23:58:11.160141+0000","last_became_peered":"2024-09-15T23:58:11.160141+0000","last_unstale":"2024-09-15T23:58:12.153498+0000","last_undegraded":"2024-09-15T23:58:12.153498+0000","last_fullsized":"2024-09-15T23:58:12.153498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:36.730764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148654+0000","last_change":"2024-09-15T23:58:11.153493+0000","last_active":"2024-09-15T23:58:12.148654+0000","last_peered":"2024-09-15T23:58:12.148654+0000","last_clean":"2024-09-15T23:58:12.148654+0000","last_became_active":"2024-09-15T23:58:11.153026+0000","last_became_peered":"2024-09-15T23:58:11.153026+0000","last_unstale":"2024-09-15T23:58:12.148654+0000","last_undegraded":"2024-09-15T23:58:12.148654+0000","last_fullsized":"2024-09-15T23:58:12.148654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:15.700186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148212+0000","last_change":"2024-09-15T23:58:11.159991+0000","last_active":"2024-09-15T23:58:12.148212+0000","last_peered":"2024-09-15T23:58:12.148212+0000","last_clean":"2024-09-15T23:58:12.148212+0000","last_became_active":"2024-09-15T23:58:11.159656+0000","last_became_peered":"2024-09-15T23:58:11.159656+0000","last_unstale":"2024-09-15T23:58:12.148212+0000","last_undegraded":"2024-09-15T23:58:12.148212+0000","last_fullsized":"2024-09-15T23:58:12.148212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:53.065297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136354+0000","last_change":"2024-09-15T23:58:11.154058+0000","last_active":"2024-09-15T23:58:12.136354+0000","last_peered":"2024-09-15T23:58:12.136354+0000","last_clean":"2024-09-15T23:58:12.136354+0000","last_became_active":"2024-09-15T23:58:11.153769+0000","last_became_peered":"2024-09-15T23:58:11.153769+0000","last_unstale":"2024-09-15T23:58:12.136354+0000","last_undegraded":"2024-09-15T23:58:12.136354+0000","last_fullsized":"2024-09-15T23:58:12.136354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:03:54.768080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136887+0000","last_change":"2024-09-15T23:58:11.148173+0000","last_active":"2024-09-15T23:58:12.136887+0000","last_peered":"2024-09-15T23:58:12.136887+0000","last_clean":"2024-09-15T23:58:12.136887+0000","last_became_active":"2024-09-15T23:58:11.147843+0000","last_became_peered":"2024-09-15T23:58:11.147843+0000","last_unstale":"2024-09-15T23:58:12.136887+0000","last_undegraded":"2024-09-15T23:58:12.136887+0000","last_fullsized":"2024-09-15T23:58:12.136887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:29.789888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136563+0000","last_change":"2024-09-15T23:58:11.162113+0000","last_active":"2024-09-15T23:58:12.136563+0000","last_peered":"2024-09-15T23:58:12.136563+0000","last_clean":"2024-09-15T23:58:12.136563+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.136563+0000","last_undegraded":"2024-09-15T23:58:12.136563+0000","last_fullsized":"2024-09-15T23:58:12.136563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:05.334727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148712+0000","last_change":"2024-09-15T23:58:11.150172+0000","last_active":"2024-09-15T23:58:12.148712+0000","last_peered":"2024-09-15T23:58:12.148712+0000","last_clean":"2024-09-15T23:58:12.148712+0000","last_became_active":"2024-09-15T23:58:11.149800+0000","last_became_peered":"2024-09-15T23:58:11.149800+0000","last_unstale":"2024-09-15T23:58:12.148712+0000","last_undegraded":"2024-09-15T23:58:12.148712+0000","last_fullsized":"2024-09-15T23:58:12.148712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:16.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154255+0000","last_change":"2024-09-15T23:58:11.153829+0000","last_active":"2024-09-15T23:58:12.154255+0000","last_peered":"2024-09-15T23:58:12.154255+0000","last_clean":"2024-09-15T23:58:12.154255+0000","last_became_active":"2024-09-15T23:58:11.153512+0000","last_became_peered":"2024-09-15T23:58:11.153512+0000","last_unstale":"2024-09-15T23:58:12.154255+0000","last_undegraded":"2024-09-15T23:58:12.154255+0000","last_fullsized":"2024-09-15T23:58:12.154255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:36.405204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147909+0000","last_change":"2024-09-15T23:58:11.162122+0000","last_active":"2024-09-15T23:58:12.147909+0000","last_peered":"2024-09-15T23:58:12.147909+0000","last_clean":"2024-09-15T23:58:12.147909+0000","last_became_active":"2024-09-15T23:58:11.161814+0000","last_became_peered":"2024-09-15T23:58:11.161814+0000","last_unstale":"2024-09-15T23:58:12.147909+0000","last_undegraded":"2024-09-15T23:58:12.147909+0000","last_fullsized":"2024-09-15T23:58:12.147909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:07.058458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150959+0000","last_change":"2024-09-15T23:58:11.157204+0000","last_active":"2024-09-15T23:58:12.150959+0000","last_peered":"2024-09-15T23:58:12.150959+0000","last_clean":"2024-09-15T23:58:12.150959+0000","last_became_active":"2024-09-15T23:58:11.156906+0000","last_became_peered":"2024-09-15T23:58:11.156906+0000","last_unstale":"2024-09-15T23:58:12.150959+0000","last_undegraded":"2024-09-15T23:58:12.150959+0000","last_fullsized":"2024-09-15T23:58:12.150959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:04.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148236+0000","last_change":"2024-09-15T23:58:11.150191+0000","last_active":"2024-09-15T23:58:12.148236+0000","last_peered":"2024-09-15T23:58:12.148236+0000","last_clean":"2024-09-15T23:58:12.148236+0000","last_became_active":"2024-09-15T23:58:11.149616+0000","last_became_peered":"2024-09-15T23:58:11.149616+0000","last_unstale":"2024-09-15T23:58:12.148236+0000","last_undegraded":"2024-09-15T23:58:12.148236+0000","last_fullsized":"2024-09-15T23:58:12.148236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:49.956626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148515+0000","last_change":"2024-09-15T23:58:11.162127+0000","last_active":"2024-09-15T23:58:12.148515+0000","last_peered":"2024-09-15T23:58:12.148515+0000","last_clean":"2024-09-15T23:58:12.148515+0000","last_became_active":"2024-09-15T23:58:11.161766+0000","last_became_peered":"2024-09-15T23:58:11.161766+0000","last_unstale":"2024-09-15T23:58:12.148515+0000","last_undegraded":"2024-09-15T23:58:12.148515+0000","last_fullsized":"2024-09-15T23:58:12.148515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:09.626747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154236+0000","last_change":"2024-09-15T23:58:11.153546+0000","last_active":"2024-09-15T23:58:12.154236+0000","last_peered":"2024-09-15T23:58:12.154236+0000","last_clean":"2024-09-15T23:58:12.154236+0000","last_became_active":"2024-09-15T23:58:11.153116+0000","last_became_peered":"2024-09-15T23:58:11.153116+0000","last_unstale":"2024-09-15T23:58:12.154236+0000","last_undegraded":"2024-09-15T23:58:12.154236+0000","last_fullsized":"2024-09-15T23:58:12.154236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:02.389068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148125+0000","last_change":"2024-09-15T23:58:11.147446+0000","last_active":"2024-09-15T23:58:12.148125+0000","last_peered":"2024-09-15T23:58:12.148125+0000","last_clean":"2024-09-15T23:58:12.148125+0000","last_became_active":"2024-09-15T23:58:11.146285+0000","last_became_peered":"2024-09-15T23:58:11.146285+0000","last_unstale":"2024-09-15T23:58:12.148125+0000","last_undegraded":"2024-09-15T23:58:12.148125+0000","last_fullsized":"2024-09-15T23:58:12.148125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:38.741814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:32.950663+0000","last_change":"2024-09-15T23:58:09.123719+0000","last_active":"2024-09-15T23:58:32.950663+0000","last_peered":"2024-09-15T23:58:32.950663+0000","last_clean":"2024-09-15T23:58:32.950663+0000","last_became_active":"2024-09-15T23:58:09.123082+0000","last_became_peered":"2024-09-15T23:58:09.123082+0000","last_unstale":"2024-09-15T23:58:32.950663+0000","last_undegraded":"2024-09-15T23:58:32.950663+0000","last_fullsized":"2024-09-15T23:58:32.950663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:46.704121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137195+0000","last_change":"2024-09-15T23:58:11.162571+0000","last_active":"2024-09-15T23:58:12.137195+0000","last_peered":"2024-09-15T23:58:12.137195+0000","last_clean":"2024-09-15T23:58:12.137195+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.137195+0000","last_undegraded":"2024-09-15T23:58:12.137195+0000","last_fullsized":"2024-09-15T23:58:12.137195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:58.134589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.290419+0000","last_change":"2024-09-15T23:58:09.125683+0000","last_active":"2024-09-15T23:58:22.290419+0000","last_peered":"2024-09-15T23:58:22.290419+0000","last_clean":"2024-09-15T23:58:22.290419+0000","last_became_active":"2024-09-15T23:58:09.125345+0000","last_became_peered":"2024-09-15T23:58:09.125345+0000","last_unstale":"2024-09-15T23:58:22.290419+0000","last_undegraded":"2024-09-15T23:58:22.290419+0000","last_fullsized":"2024-09-15T23:58:22.290419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:39.998850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284754+0000","last_change":"2024-09-15T23:58:09.116886+0000","last_active":"2024-09-15T23:58:21.284754+0000","last_peered":"2024-09-15T23:58:21.284754+0000","last_clean":"2024-09-15T23:58:21.284754+0000","last_became_active":"2024-09-15T23:58:09.116510+0000","last_became_peered":"2024-09-15T23:58:09.116510+0000","last_unstale":"2024-09-15T23:58:21.284754+0000","last_undegraded":"2024-09-15T23:58:21.284754+0000","last_fullsized":"2024-09-15T23:58:21.284754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:10.368106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136363+0000","last_change":"2024-09-15T23:58:11.144759+0000","last_active":"2024-09-15T23:58:12.136363+0000","last_peered":"2024-09-15T23:58:12.136363+0000","last_clean":"2024-09-15T23:58:12.136363+0000","last_became_active":"2024-09-15T23:58:11.144435+0000","last_became_peered":"2024-09-15T23:58:11.144435+0000","last_unstale":"2024-09-15T23:58:12.136363+0000","last_undegraded":"2024-09-15T23:58:12.136363+0000","last_fullsized":"2024-09-15T23:58:12.136363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:32.787727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300459+0000","last_change":"2024-09-15T23:58:09.124938+0000","last_active":"2024-09-15T23:58:23.300459+0000","last_peered":"2024-09-15T23:58:23.300459+0000","last_clean":"2024-09-15T23:58:23.300459+0000","last_became_active":"2024-09-15T23:58:09.124532+0000","last_became_peered":"2024-09-15T23:58:09.124532+0000","last_unstale":"2024-09-15T23:58:23.300459+0000","last_undegraded":"2024-09-15T23:58:23.300459+0000","last_fullsized":"2024-09-15T23:58:23.300459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148543+0000","last_change":"2024-09-15T23:58:11.145940+0000","last_active":"2024-09-15T23:58:12.148543+0000","last_peered":"2024-09-15T23:58:12.148543+0000","last_clean":"2024-09-15T23:58:12.148543+0000","last_became_active":"2024-09-15T23:58:11.145642+0000","last_became_peered":"2024-09-15T23:58:11.145642+0000","last_unstale":"2024-09-15T23:58:12.148543+0000","last_undegraded":"2024-09-15T23:58:12.148543+0000","last_fullsized":"2024-09-15T23:58:12.148543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:58.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:31.948519+0000","last_change":"2024-09-15T23:58:09.126742+0000","last_active":"2024-09-15T23:58:31.948519+0000","last_peered":"2024-09-15T23:58:31.948519+0000","last_clean":"2024-09-15T23:58:31.948519+0000","last_became_active":"2024-09-15T23:58:09.126432+0000","last_became_peered":"2024-09-15T23:58:09.126432+0000","last_unstale":"2024-09-15T23:58:31.948519+0000","last_undegraded":"2024-09-15T23:58:31.948519+0000","last_fullsized":"2024-09-15T23:58:31.948519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:11.582346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148667+0000","last_change":"2024-09-15T23:58:11.150130+0000","last_active":"2024-09-15T23:58:12.148667+0000","last_peered":"2024-09-15T23:58:12.148667+0000","last_clean":"2024-09-15T23:58:12.148667+0000","last_became_active":"2024-09-15T23:58:11.149531+0000","last_became_peered":"2024-09-15T23:58:11.149531+0000","last_unstale":"2024-09-15T23:58:12.148667+0000","last_undegraded":"2024-09-15T23:58:12.148667+0000","last_fullsized":"2024-09-15T23:58:12.148667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:07.206505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299397+0000","last_change":"2024-09-15T23:58:09.121229+0000","last_active":"2024-09-15T23:58:23.299397+0000","last_peered":"2024-09-15T23:58:23.299397+0000","last_clean":"2024-09-15T23:58:23.299397+0000","last_became_active":"2024-09-15T23:58:09.120808+0000","last_became_peered":"2024-09-15T23:58:09.120808+0000","last_unstale":"2024-09-15T23:58:23.299397+0000","last_undegraded":"2024-09-15T23:58:23.299397+0000","last_fullsized":"2024-09-15T23:58:23.299397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:34.155774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151252+0000","last_change":"2024-09-15T23:58:11.157253+0000","last_active":"2024-09-15T23:58:12.151252+0000","last_peered":"2024-09-15T23:58:12.151252+0000","last_clean":"2024-09-15T23:58:12.151252+0000","last_became_active":"2024-09-15T23:58:11.156952+0000","last_became_peered":"2024-09-15T23:58:11.156952+0000","last_unstale":"2024-09-15T23:58:12.151252+0000","last_undegraded":"2024-09-15T23:58:12.151252+0000","last_fullsized":"2024-09-15T23:58:12.151252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:09.840371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289729+0000","last_change":"2024-09-15T23:58:09.322776+0000","last_active":"2024-09-15T23:58:22.289729+0000","last_peered":"2024-09-15T23:58:22.289729+0000","last_clean":"2024-09-15T23:58:22.289729+0000","last_became_active":"2024-09-15T23:58:09.322304+0000","last_became_peered":"2024-09-15T23:58:09.322304+0000","last_unstale":"2024-09-15T23:58:22.289729+0000","last_undegraded":"2024-09-15T23:58:22.289729+0000","last_fullsized":"2024-09-15T23:58:22.289729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:51.413578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148251+0000","last_change":"2024-09-15T23:58:11.165067+0000","last_active":"2024-09-15T23:58:12.148251+0000","last_peered":"2024-09-15T23:58:12.148251+0000","last_clean":"2024-09-15T23:58:12.148251+0000","last_became_active":"2024-09-15T23:58:11.163722+0000","last_became_peered":"2024-09-15T23:58:11.163722+0000","last_unstale":"2024-09-15T23:58:12.148251+0000","last_undegraded":"2024-09-15T23:58:12.148251+0000","last_fullsized":"2024-09-15T23:58:12.148251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:01.354822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'23","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:31.949069+0000","last_change":"2024-09-15T23:58:09.128077+0000","last_active":"2024-09-15T23:58:31.949069+0000","last_peered":"2024-09-15T23:58:31.949069+0000","last_clean":"2024-09-15T23:58:31.949069+0000","last_became_active":"2024-09-15T23:58:09.127215+0000","last_became_peered":"2024-09-15T23:58:09.127215+0000","last_unstale":"2024-09-15T23:58:31.949069+0000","last_undegraded":"2024-09-15T23:58:31.949069+0000","last_fullsized":"2024-09-15T23:58:31.949069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:47.875279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":51,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150490+0000","last_change":"2024-09-15T23:58:11.156020+0000","last_active":"2024-09-15T23:58:12.150490+0000","last_peered":"2024-09-15T23:58:12.150490+0000","last_clean":"2024-09-15T23:58:12.150490+0000","last_became_active":"2024-09-15T23:58:11.155634+0000","last_became_peered":"2024-09-15T23:58:11.155634+0000","last_unstale":"2024-09-15T23:58:12.150490+0000","last_undegraded":"2024-09-15T23:58:12.150490+0000","last_fullsized":"2024-09-15T23:58:12.150490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934453+0000","last_change":"2024-09-15T23:58:09.123084+0000","last_active":"2024-09-15T23:58:23.934453+0000","last_peered":"2024-09-15T23:58:23.934453+0000","last_clean":"2024-09-15T23:58:23.934453+0000","last_became_active":"2024-09-15T23:58:09.122594+0000","last_became_peered":"2024-09-15T23:58:09.122594+0000","last_unstale":"2024-09-15T23:58:23.934453+0000","last_undegraded":"2024-09-15T23:58:23.934453+0000","last_fullsized":"2024-09-15T23:58:23.934453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:47.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149295+0000","last_change":"2024-09-15T23:58:11.155082+0000","last_active":"2024-09-15T23:58:12.149295+0000","last_peered":"2024-09-15T23:58:12.149295+0000","last_clean":"2024-09-15T23:58:12.149295+0000","last_became_active":"2024-09-15T23:58:11.154764+0000","last_became_peered":"2024-09-15T23:58:11.154764+0000","last_unstale":"2024-09-15T23:58:12.149295+0000","last_undegraded":"2024-09-15T23:58:12.149295+0000","last_fullsized":"2024-09-15T23:58:12.149295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:47.170383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300267+0000","last_change":"2024-09-15T23:58:09.124416+0000","last_active":"2024-09-15T23:58:23.300267+0000","last_peered":"2024-09-15T23:58:23.300267+0000","last_clean":"2024-09-15T23:58:23.300267+0000","last_became_active":"2024-09-15T23:58:09.123836+0000","last_became_peered":"2024-09-15T23:58:09.123836+0000","last_unstale":"2024-09-15T23:58:23.300267+0000","last_undegraded":"2024-09-15T23:58:23.300267+0000","last_fullsized":"2024-09-15T23:58:23.300267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:35.730677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150934+0000","last_change":"2024-09-15T23:58:11.162220+0000","last_active":"2024-09-15T23:58:12.150934+0000","last_peered":"2024-09-15T23:58:12.150934+0000","last_clean":"2024-09-15T23:58:12.150934+0000","last_became_active":"2024-09-15T23:58:11.161887+0000","last_became_peered":"2024-09-15T23:58:11.161887+0000","last_unstale":"2024-09-15T23:58:12.150934+0000","last_undegraded":"2024-09-15T23:58:12.150934+0000","last_fullsized":"2024-09-15T23:58:12.150934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:35.070135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:33.951146+0000","last_change":"2024-09-15T23:58:09.123010+0000","last_active":"2024-09-15T23:58:33.951146+0000","last_peered":"2024-09-15T23:58:33.951146+0000","last_clean":"2024-09-15T23:58:33.951146+0000","last_became_active":"2024-09-15T23:58:09.122615+0000","last_became_peered":"2024-09-15T23:58:09.122615+0000","last_unstale":"2024-09-15T23:58:33.951146+0000","last_undegraded":"2024-09-15T23:58:33.951146+0000","last_fullsized":"2024-09-15T23:58:33.951146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:09.634409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150261+0000","last_change":"2024-09-15T23:58:11.159069+0000","last_active":"2024-09-15T23:58:12.150261+0000","last_peered":"2024-09-15T23:58:12.150261+0000","last_clean":"2024-09-15T23:58:12.150261+0000","last_became_active":"2024-09-15T23:58:11.158659+0000","last_became_peered":"2024-09-15T23:58:11.158659+0000","last_unstale":"2024-09-15T23:58:12.150261+0000","last_undegraded":"2024-09-15T23:58:12.150261+0000","last_fullsized":"2024-09-15T23:58:12.150261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:24.336089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298401+0000","last_change":"2024-09-15T23:58:09.322903+0000","last_active":"2024-09-15T23:58:23.298401+0000","last_peered":"2024-09-15T23:58:23.298401+0000","last_clean":"2024-09-15T23:58:23.298401+0000","last_became_active":"2024-09-15T23:58:09.322326+0000","last_became_peered":"2024-09-15T23:58:09.322326+0000","last_unstale":"2024-09-15T23:58:23.298401+0000","last_undegraded":"2024-09-15T23:58:23.298401+0000","last_fullsized":"2024-09-15T23:58:23.298401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:58.749293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136919+0000","last_change":"2024-09-15T23:58:11.162104+0000","last_active":"2024-09-15T23:58:12.136919+0000","last_peered":"2024-09-15T23:58:12.136919+0000","last_clean":"2024-09-15T23:58:12.136919+0000","last_became_active":"2024-09-15T23:58:11.161739+0000","last_became_peered":"2024-09-15T23:58:11.161739+0000","last_unstale":"2024-09-15T23:58:12.136919+0000","last_undegraded":"2024-09-15T23:58:12.136919+0000","last_fullsized":"2024-09-15T23:58:12.136919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:55.316245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.279815+0000","last_change":"2024-09-15T23:58:09.119975+0000","last_active":"2024-09-15T23:58:22.279815+0000","last_peered":"2024-09-15T23:58:22.279815+0000","last_clean":"2024-09-15T23:58:22.279815+0000","last_became_active":"2024-09-15T23:58:09.119340+0000","last_became_peered":"2024-09-15T23:58:09.119340+0000","last_unstale":"2024-09-15T23:58:22.279815+0000","last_undegraded":"2024-09-15T23:58:22.279815+0000","last_fullsized":"2024-09-15T23:58:22.279815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:22:14.858076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153874+0000","last_change":"2024-09-15T23:58:11.157389+0000","last_active":"2024-09-15T23:58:12.153874+0000","last_peered":"2024-09-15T23:58:12.153874+0000","last_clean":"2024-09-15T23:58:12.153874+0000","last_became_active":"2024-09-15T23:58:11.157033+0000","last_became_peered":"2024-09-15T23:58:11.157033+0000","last_unstale":"2024-09-15T23:58:12.153874+0000","last_undegraded":"2024-09-15T23:58:12.153874+0000","last_fullsized":"2024-09-15T23:58:12.153874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:49.149860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299541+0000","last_change":"2024-09-15T23:58:09.120937+0000","last_active":"2024-09-15T23:58:23.299541+0000","last_peered":"2024-09-15T23:58:23.299541+0000","last_clean":"2024-09-15T23:58:23.299541+0000","last_became_active":"2024-09-15T23:58:09.120586+0000","last_became_peered":"2024-09-15T23:58:09.120586+0000","last_unstale":"2024-09-15T23:58:23.299541+0000","last_undegraded":"2024-09-15T23:58:23.299541+0000","last_fullsized":"2024-09-15T23:58:23.299541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:18.432631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150943+0000","last_change":"2024-09-15T23:58:11.155316+0000","last_active":"2024-09-15T23:58:12.150943+0000","last_peered":"2024-09-15T23:58:12.150943+0000","last_clean":"2024-09-15T23:58:12.150943+0000","last_became_active":"2024-09-15T23:58:11.154975+0000","last_became_peered":"2024-09-15T23:58:11.154975+0000","last_unstale":"2024-09-15T23:58:12.150943+0000","last_undegraded":"2024-09-15T23:58:12.150943+0000","last_fullsized":"2024-09-15T23:58:12.150943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:09.856126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300569+0000","last_change":"2024-09-15T23:58:09.125045+0000","last_active":"2024-09-15T23:58:23.300569+0000","last_peered":"2024-09-15T23:58:23.300569+0000","last_clean":"2024-09-15T23:58:23.300569+0000","last_became_active":"2024-09-15T23:58:09.124577+0000","last_became_peered":"2024-09-15T23:58:09.124577+0000","last_unstale":"2024-09-15T23:58:23.300569+0000","last_undegraded":"2024-09-15T23:58:23.300569+0000","last_fullsized":"2024-09-15T23:58:23.300569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:48.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150843+0000","last_change":"2024-09-15T23:58:11.160241+0000","last_active":"2024-09-15T23:58:12.150843+0000","last_peered":"2024-09-15T23:58:12.150843+0000","last_clean":"2024-09-15T23:58:12.150843+0000","last_became_active":"2024-09-15T23:58:11.159886+0000","last_became_peered":"2024-09-15T23:58:11.159886+0000","last_unstale":"2024-09-15T23:58:12.150843+0000","last_undegraded":"2024-09-15T23:58:12.150843+0000","last_fullsized":"2024-09-15T23:58:12.150843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:02.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:30.947866+0000","last_change":"2024-09-15T23:58:09.482888+0000","last_active":"2024-09-15T23:58:30.947866+0000","last_peered":"2024-09-15T23:58:30.947866+0000","last_clean":"2024-09-15T23:58:30.947866+0000","last_became_active":"2024-09-15T23:58:09.482497+0000","last_became_peered":"2024-09-15T23:58:09.482497+0000","last_unstale":"2024-09-15T23:58:30.947866+0000","last_undegraded":"2024-09-15T23:58:30.947866+0000","last_fullsized":"2024-09-15T23:58:30.947866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:59.320794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151266+0000","last_change":"2024-09-15T23:58:11.159452+0000","last_active":"2024-09-15T23:58:12.151266+0000","last_peered":"2024-09-15T23:58:12.151266+0000","last_clean":"2024-09-15T23:58:12.151266+0000","last_became_active":"2024-09-15T23:58:11.157610+0000","last_became_peered":"2024-09-15T23:58:11.157610+0000","last_unstale":"2024-09-15T23:58:12.151266+0000","last_undegraded":"2024-09-15T23:58:12.151266+0000","last_fullsized":"2024-09-15T23:58:12.151266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:17.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150373+0000","last_change":"2024-09-15T23:56:56.876989+0000","last_active":"2024-09-15T23:58:12.150373+0000","last_peered":"2024-09-15T23:58:12.150373+0000","last_clean":"2024-09-15T23:58:12.150373+0000","last_became_active":"2024-09-15T23:56:56.876413+0000","last_became_peered":"2024-09-15T23:56:56.876413+0000","last_unstale":"2024-09-15T23:58:12.150373+0000","last_undegraded":"2024-09-15T23:58:12.150373+0000","last_fullsized":"2024-09-15T23:58:12.150373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_clean_scrub_stamp":"2024-09-15T23:53:27.674229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:06.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934257+0000","last_change":"2024-09-15T23:58:09.124514+0000","last_active":"2024-09-15T23:58:23.934257+0000","last_peered":"2024-09-15T23:58:23.934257+0000","last_clean":"2024-09-15T23:58:23.934257+0000","last_became_active":"2024-09-15T23:58:09.122124+0000","last_became_peered":"2024-09-15T23:58:09.122124+0000","last_unstale":"2024-09-15T23:58:23.934257+0000","last_undegraded":"2024-09-15T23:58:23.934257+0000","last_fullsized":"2024-09-15T23:58:23.934257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:13.563073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136568+0000","last_change":"2024-09-15T23:58:11.149559+0000","last_active":"2024-09-15T23:58:12.136568+0000","last_peered":"2024-09-15T23:58:12.136568+0000","last_clean":"2024-09-15T23:58:12.136568+0000","last_became_active":"2024-09-15T23:58:11.149215+0000","last_became_peered":"2024-09-15T23:58:11.149215+0000","last_unstale":"2024-09-15T23:58:12.136568+0000","last_undegraded":"2024-09-15T23:58:12.136568+0000","last_fullsized":"2024-09-15T23:58:12.136568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:16.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284817+0000","last_change":"2024-09-15T23:58:09.127770+0000","last_active":"2024-09-15T23:58:21.284817+0000","last_peered":"2024-09-15T23:58:21.284817+0000","last_clean":"2024-09-15T23:58:21.284817+0000","last_became_active":"2024-09-15T23:58:09.127404+0000","last_became_peered":"2024-09-15T23:58:09.127404+0000","last_unstale":"2024-09-15T23:58:21.284817+0000","last_undegraded":"2024-09-15T23:58:21.284817+0000","last_fullsized":"2024-09-15T23:58:21.284817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:48.176290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148872+0000","last_change":"2024-09-15T23:58:11.162161+0000","last_active":"2024-09-15T23:58:12.148872+0000","last_peered":"2024-09-15T23:58:12.148872+0000","last_clean":"2024-09-15T23:58:12.148872+0000","last_became_active":"2024-09-15T23:58:11.161879+0000","last_became_peered":"2024-09-15T23:58:11.161879+0000","last_unstale":"2024-09-15T23:58:12.148872+0000","last_undegraded":"2024-09-15T23:58:12.148872+0000","last_fullsized":"2024-09-15T23:58:12.148872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:34.951818+0000","last_change":"2024-09-15T23:58:09.322611+0000","last_active":"2024-09-15T23:58:34.951818+0000","last_peered":"2024-09-15T23:58:34.951818+0000","last_clean":"2024-09-15T23:58:34.951818+0000","last_became_active":"2024-09-15T23:58:09.322254+0000","last_became_peered":"2024-09-15T23:58:09.322254+0000","last_unstale":"2024-09-15T23:58:34.951818+0000","last_undegraded":"2024-09-15T23:58:34.951818+0000","last_fullsized":"2024-09-15T23:58:34.951818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:03.803927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137151+0000","last_change":"2024-09-15T23:58:11.152219+0000","last_active":"2024-09-15T23:58:12.137151+0000","last_peered":"2024-09-15T23:58:12.137151+0000","last_clean":"2024-09-15T23:58:12.137151+0000","last_became_active":"2024-09-15T23:58:11.151885+0000","last_became_peered":"2024-09-15T23:58:11.151885+0000","last_unstale":"2024-09-15T23:58:12.137151+0000","last_undegraded":"2024-09-15T23:58:12.137151+0000","last_fullsized":"2024-09-15T23:58:12.137151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:10.780163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:34.952053+0000","last_change":"2024-09-15T23:58:09.125152+0000","last_active":"2024-09-15T23:58:34.952053+0000","last_peered":"2024-09-15T23:58:34.952053+0000","last_clean":"2024-09-15T23:58:34.952053+0000","last_became_active":"2024-09-15T23:58:09.124754+0000","last_became_peered":"2024-09-15T23:58:09.124754+0000","last_unstale":"2024-09-15T23:58:34.952053+0000","last_undegraded":"2024-09-15T23:58:34.952053+0000","last_fullsized":"2024-09-15T23:58:34.952053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:02.094909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136921+0000","last_change":"2024-09-15T23:58:11.160257+0000","last_active":"2024-09-15T23:58:12.136921+0000","last_peered":"2024-09-15T23:58:12.136921+0000","last_clean":"2024-09-15T23:58:12.136921+0000","last_became_active":"2024-09-15T23:58:11.159868+0000","last_became_peered":"2024-09-15T23:58:11.159868+0000","last_unstale":"2024-09-15T23:58:12.136921+0000","last_undegraded":"2024-09-15T23:58:12.136921+0000","last_fullsized":"2024-09-15T23:58:12.136921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.528002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.226786+0000","last_change":"2024-09-15T23:58:09.323034+0000","last_active":"2024-09-15T23:58:12.226786+0000","last_peered":"2024-09-15T23:58:12.226786+0000","last_clean":"2024-09-15T23:58:12.226786+0000","last_became_active":"2024-09-15T23:58:09.322658+0000","last_became_peered":"2024-09-15T23:58:09.322658+0000","last_unstale":"2024-09-15T23:58:12.226786+0000","last_undegraded":"2024-09-15T23:58:12.226786+0000","last_fullsized":"2024-09-15T23:58:12.226786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:20:11.175093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.862670+0000","last_change":"2024-09-15T23:58:11.149216+0000","last_active":"2024-09-15T23:58:12.862670+0000","last_peered":"2024-09-15T23:58:12.862670+0000","last_clean":"2024-09-15T23:58:12.862670+0000","last_became_active":"2024-09-15T23:58:11.148893+0000","last_became_peered":"2024-09-15T23:58:11.148893+0000","last_unstale":"2024-09-15T23:58:12.862670+0000","last_undegraded":"2024-09-15T23:58:12.862670+0000","last_fullsized":"2024-09-15T23:58:12.862670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:07.749101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300019+0000","last_change":"2024-09-15T23:58:09.323030+0000","last_active":"2024-09-15T23:58:23.300019+0000","last_peered":"2024-09-15T23:58:23.300019+0000","last_clean":"2024-09-15T23:58:23.300019+0000","last_became_active":"2024-09-15T23:58:09.322569+0000","last_became_peered":"2024-09-15T23:58:09.322569+0000","last_unstale":"2024-09-15T23:58:23.300019+0000","last_undegraded":"2024-09-15T23:58:23.300019+0000","last_fullsized":"2024-09-15T23:58:23.300019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:52.811395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148063+0000","last_change":"2024-09-15T23:58:11.154084+0000","last_active":"2024-09-15T23:58:12.148063+0000","last_peered":"2024-09-15T23:58:12.148063+0000","last_clean":"2024-09-15T23:58:12.148063+0000","last_became_active":"2024-09-15T23:58:11.153720+0000","last_became_peered":"2024-09-15T23:58:11.153720+0000","last_unstale":"2024-09-15T23:58:12.148063+0000","last_undegraded":"2024-09-15T23:58:12.148063+0000","last_fullsized":"2024-09-15T23:58:12.148063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:07.468812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:29.148622+0000","last_change":"2024-09-15T23:58:09.116996+0000","last_active":"2024-09-15T23:58:29.148622+0000","last_peered":"2024-09-15T23:58:29.148622+0000","last_clean":"2024-09-15T23:58:29.148622+0000","last_became_active":"2024-09-15T23:58:09.116531+0000","last_became_peered":"2024-09-15T23:58:09.116531+0000","last_unstale":"2024-09-15T23:58:29.148622+0000","last_undegraded":"2024-09-15T23:58:29.148622+0000","last_fullsized":"2024-09-15T23:58:29.148622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.094719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136595+0000","last_change":"2024-09-15T23:58:11.155704+0000","last_active":"2024-09-15T23:58:12.136595+0000","last_peered":"2024-09-15T23:58:12.136595+0000","last_clean":"2024-09-15T23:58:12.136595+0000","last_became_active":"2024-09-15T23:58:11.155344+0000","last_became_peered":"2024-09-15T23:58:11.155344+0000","last_unstale":"2024-09-15T23:58:12.136595+0000","last_undegraded":"2024-09-15T23:58:12.136595+0000","last_fullsized":"2024-09-15T23:58:12.136595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:50.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298948+0000","last_change":"2024-09-15T23:58:09.482837+0000","last_active":"2024-09-15T23:58:23.298948+0000","last_peered":"2024-09-15T23:58:23.298948+0000","last_clean":"2024-09-15T23:58:23.298948+0000","last_became_active":"2024-09-15T23:58:09.482444+0000","last_became_peered":"2024-09-15T23:58:09.482444+0000","last_unstale":"2024-09-15T23:58:23.298948+0000","last_undegraded":"2024-09-15T23:58:23.298948+0000","last_fullsized":"2024-09-15T23:58:23.298948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:05.962471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150939+0000","last_change":"2024-09-15T23:58:11.157249+0000","last_active":"2024-09-15T23:58:12.150939+0000","last_peered":"2024-09-15T23:58:12.150939+0000","last_clean":"2024-09-15T23:58:12.150939+0000","last_became_active":"2024-09-15T23:58:11.156871+0000","last_became_peered":"2024-09-15T23:58:11.156871+0000","last_unstale":"2024-09-15T23:58:12.150939+0000","last_undegraded":"2024-09-15T23:58:12.150939+0000","last_fullsized":"2024-09-15T23:58:12.150939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:16.986165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279567+0000","last_change":"2024-09-15T23:58:09.324320+0000","last_active":"2024-09-15T23:58:20.279567+0000","last_peered":"2024-09-15T23:58:20.279567+0000","last_clean":"2024-09-15T23:58:20.279567+0000","last_became_active":"2024-09-15T23:58:09.323996+0000","last_became_peered":"2024-09-15T23:58:09.323996+0000","last_unstale":"2024-09-15T23:58:20.279567+0000","last_undegraded":"2024-09-15T23:58:20.279567+0000","last_fullsized":"2024-09-15T23:58:20.279567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:34.456229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147904+0000","last_change":"2024-09-15T23:58:11.146836+0000","last_active":"2024-09-15T23:58:12.147904+0000","last_peered":"2024-09-15T23:58:12.147904+0000","last_clean":"2024-09-15T23:58:12.147904+0000","last_became_active":"2024-09-15T23:58:11.146470+0000","last_became_peered":"2024-09-15T23:58:11.146470+0000","last_unstale":"2024-09-15T23:58:12.147904+0000","last_undegraded":"2024-09-15T23:58:12.147904+0000","last_fullsized":"2024-09-15T23:58:12.147904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.721954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300504+0000","last_change":"2024-09-15T23:58:09.121920+0000","last_active":"2024-09-15T23:58:23.300504+0000","last_peered":"2024-09-15T23:58:23.300504+0000","last_clean":"2024-09-15T23:58:23.300504+0000","last_became_active":"2024-09-15T23:58:09.121605+0000","last_became_peered":"2024-09-15T23:58:09.121605+0000","last_unstale":"2024-09-15T23:58:23.300504+0000","last_undegraded":"2024-09-15T23:58:23.300504+0000","last_fullsized":"2024-09-15T23:58:23.300504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:12.707690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154389+0000","last_change":"2024-09-15T23:58:11.153676+0000","last_active":"2024-09-15T23:58:12.154389+0000","last_peered":"2024-09-15T23:58:12.154389+0000","last_clean":"2024-09-15T23:58:12.154389+0000","last_became_active":"2024-09-15T23:58:11.152866+0000","last_became_peered":"2024-09-15T23:58:11.152866+0000","last_unstale":"2024-09-15T23:58:12.154389+0000","last_undegraded":"2024-09-15T23:58:12.154389+0000","last_fullsized":"2024-09-15T23:58:12.154389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:02.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289656+0000","last_change":"2024-09-15T23:58:09.121440+0000","last_active":"2024-09-15T23:58:22.289656+0000","last_peered":"2024-09-15T23:58:22.289656+0000","last_clean":"2024-09-15T23:58:22.289656+0000","last_became_active":"2024-09-15T23:58:09.121006+0000","last_became_peered":"2024-09-15T23:58:09.121006+0000","last_unstale":"2024-09-15T23:58:22.289656+0000","last_undegraded":"2024-09-15T23:58:22.289656+0000","last_fullsized":"2024-09-15T23:58:22.289656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:32.584222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136632+0000","last_change":"2024-09-15T23:58:11.155311+0000","last_active":"2024-09-15T23:58:12.136632+0000","last_peered":"2024-09-15T23:58:12.136632+0000","last_clean":"2024-09-15T23:58:12.136632+0000","last_became_active":"2024-09-15T23:58:11.154954+0000","last_became_peered":"2024-09-15T23:58:11.154954+0000","last_unstale":"2024-09-15T23:58:12.136632+0000","last_undegraded":"2024-09-15T23:58:12.136632+0000","last_fullsized":"2024-09-15T23:58:12.136632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:52.131451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279891+0000","last_change":"2024-09-15T23:58:09.126143+0000","last_active":"2024-09-15T23:58:20.279891+0000","last_peered":"2024-09-15T23:58:20.279891+0000","last_clean":"2024-09-15T23:58:20.279891+0000","last_became_active":"2024-09-15T23:58:09.125785+0000","last_became_peered":"2024-09-15T23:58:09.125785+0000","last_unstale":"2024-09-15T23:58:20.279891+0000","last_undegraded":"2024-09-15T23:58:20.279891+0000","last_fullsized":"2024-09-15T23:58:20.279891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:22.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150690+0000","last_change":"2024-09-15T23:58:11.160834+0000","last_active":"2024-09-15T23:58:12.150690+0000","last_peered":"2024-09-15T23:58:12.150690+0000","last_clean":"2024-09-15T23:58:12.150690+0000","last_became_active":"2024-09-15T23:58:11.160458+0000","last_became_peered":"2024-09-15T23:58:11.160458+0000","last_unstale":"2024-09-15T23:58:12.150690+0000","last_undegraded":"2024-09-15T23:58:12.150690+0000","last_fullsized":"2024-09-15T23:58:12.150690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:36.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137347+0000","last_change":"2024-09-15T23:58:11.151745+0000","last_active":"2024-09-15T23:58:12.137347+0000","last_peered":"2024-09-15T23:58:12.137347+0000","last_clean":"2024-09-15T23:58:12.137347+0000","last_became_active":"2024-09-15T23:58:11.150864+0000","last_became_peered":"2024-09-15T23:58:11.150864+0000","last_unstale":"2024-09-15T23:58:12.137347+0000","last_undegraded":"2024-09-15T23:58:12.137347+0000","last_fullsized":"2024-09-15T23:58:12.137347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:34.578125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300990+0000","last_change":"2024-09-15T23:58:09.121095+0000","last_active":"2024-09-15T23:58:23.300990+0000","last_peered":"2024-09-15T23:58:23.300990+0000","last_clean":"2024-09-15T23:58:23.300990+0000","last_became_active":"2024-09-15T23:58:09.120671+0000","last_became_peered":"2024-09-15T23:58:09.120671+0000","last_unstale":"2024-09-15T23:58:23.300990+0000","last_undegraded":"2024-09-15T23:58:23.300990+0000","last_fullsized":"2024-09-15T23:58:23.300990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:32.099810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148041+0000","last_change":"2024-09-15T23:58:11.165256+0000","last_active":"2024-09-15T23:58:12.148041+0000","last_peered":"2024-09-15T23:58:12.148041+0000","last_clean":"2024-09-15T23:58:12.148041+0000","last_became_active":"2024-09-15T23:58:11.163663+0000","last_became_peered":"2024-09-15T23:58:11.163663+0000","last_unstale":"2024-09-15T23:58:12.148041+0000","last_undegraded":"2024-09-15T23:58:12.148041+0000","last_fullsized":"2024-09-15T23:58:12.148041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:02.112109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934080+0000","last_change":"2024-09-15T23:58:09.125833+0000","last_active":"2024-09-15T23:58:23.934080+0000","last_peered":"2024-09-15T23:58:23.934080+0000","last_clean":"2024-09-15T23:58:23.934080+0000","last_became_active":"2024-09-15T23:58:09.125411+0000","last_became_peered":"2024-09-15T23:58:09.125411+0000","last_unstale":"2024-09-15T23:58:23.934080+0000","last_undegraded":"2024-09-15T23:58:23.934080+0000","last_fullsized":"2024-09-15T23:58:23.934080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:17.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151115+0000","last_change":"2024-09-15T23:58:11.159923+0000","last_active":"2024-09-15T23:58:12.151115+0000","last_peered":"2024-09-15T23:58:12.151115+0000","last_clean":"2024-09-15T23:58:12.151115+0000","last_became_active":"2024-09-15T23:58:11.159489+0000","last_became_peered":"2024-09-15T23:58:11.159489+0000","last_unstale":"2024-09-15T23:58:12.151115+0000","last_undegraded":"2024-09-15T23:58:12.151115+0000","last_fullsized":"2024-09-15T23:58:12.151115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:30.864728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153095+0000","last_change":"2024-09-15T23:58:09.322767+0000","last_active":"2024-09-15T23:58:12.153095+0000","last_peered":"2024-09-15T23:58:12.153095+0000","last_clean":"2024-09-15T23:58:12.153095+0000","last_became_active":"2024-09-15T23:58:09.322289+0000","last_became_peered":"2024-09-15T23:58:09.322289+0000","last_unstale":"2024-09-15T23:58:12.153095+0000","last_undegraded":"2024-09-15T23:58:12.153095+0000","last_fullsized":"2024-09-15T23:58:12.153095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:12.168772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136810+0000","last_change":"2024-09-15T23:58:11.153181+0000","last_active":"2024-09-15T23:58:12.136810+0000","last_peered":"2024-09-15T23:58:12.136810+0000","last_clean":"2024-09-15T23:58:12.136810+0000","last_became_active":"2024-09-15T23:58:11.152920+0000","last_became_peered":"2024-09-15T23:58:11.152920+0000","last_unstale":"2024-09-15T23:58:12.136810+0000","last_undegraded":"2024-09-15T23:58:12.136810+0000","last_fullsized":"2024-09-15T23:58:12.136810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:15.101157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:30.946575+0000","last_change":"2024-09-15T23:58:09.120470+0000","last_active":"2024-09-15T23:58:30.946575+0000","last_peered":"2024-09-15T23:58:30.946575+0000","last_clean":"2024-09-15T23:58:30.946575+0000","last_became_active":"2024-09-15T23:58:09.120143+0000","last_became_peered":"2024-09-15T23:58:09.120143+0000","last_unstale":"2024-09-15T23:58:30.946575+0000","last_undegraded":"2024-09-15T23:58:30.946575+0000","last_fullsized":"2024-09-15T23:58:30.946575+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:41.285326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47880,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":139,"num_read_kb":122,"num_write":225,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":143640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244980,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1049487,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":417323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201313,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1010221,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1009506,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":416865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":459238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":459411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":418379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:58:39.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T23:58:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:40.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:39 smithi031 ceph-mon[31203]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:40.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:39 smithi081 ceph-mon[35060]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:40.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:39 smithi102 ceph-mon[34458]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 -- 172.21.15.31:0/1659974297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c101440 msgr2=0x7f643c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 --2- 172.21.15.31:0/1659974297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c101440 0x7f643c10df30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c02a410 comp rx=0 tx=0).stop 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 -- 172.21.15.31:0/1659974297 shutdown_connections 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 --2- 172.21.15.31:0/1659974297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c10e500 0x7f643c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 --2- 172.21.15.31:0/1659974297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c101440 0x7f643c10df30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 --2- 172.21.15.31:0/1659974297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c100b20 0x7f643c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 -- 172.21.15.31:0/1659974297 >> 172.21.15.31:0/1659974297 conn(0x7f643c0fc7d0 msgr2=0x7f643c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 -- 172.21.15.31:0/1659974297 shutdown_connections 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.540+0000 7f6443943640 1 -- 172.21.15.31:0/1659974297 wait complete. 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 Processor -- start 2024-09-15T23:58:40.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 -- start start 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101440 0x7f643c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 0x7f643c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114200 con 0x7f643c100b20 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114080 con 0x7f643c10e500 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f6443943640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114380 con 0x7f643c101440 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f64416df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f64416df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59966/0 (socket says 172.21.15.31:59966) 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.541+0000 7f64416df640 1 -- 172.21.15.31:0/3509390250 learned_addr learned my addr 172.21.15.31:0/3509390250 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f6440ede640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101440 0x7f643c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 -- 172.21.15.31:0/3509390250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101440 msgr2=0x7f643c1a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f6441ee0640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 0x7f643c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:40.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101440 0x7f643c1a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 -- 172.21.15.31:0/3509390250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 msgr2=0x7f643c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 0x7f643c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 -- 172.21.15.31:0/3509390250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19aec0 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f6441ee0640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 0x7f643c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64416df640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f643800b4a0 tx=0x7f643800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438014070 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6438010ac0 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438017c30 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19b1b0 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.542+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a7500 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.543+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c06a850 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.544+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6438017070 con 0x7f643c100b20 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.545+0000 7f64327fc640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 0x7f64280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:40.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.545+0000 7f6440ede640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 0x7f64280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.545+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f64380ceed0 con 0x7f643c100b20 2024-09-15T23:58:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.545+0000 7f6440ede640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 0x7f64280a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.549+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f643809bd50 con 0x7f643c100b20 2024-09-15T23:58:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.720+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f643c062630 con 0x7f64280a3350 2024-09-15T23:58:40.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.729+0000 7f64327fc640 1 -- 172.21.15.31:0/3509390250 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258947 (secure 0 0 0) 0x7f643c062630 con 0x7f64280a3350 2024-09-15T23:58:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-15T23:58:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 msgr2=0x7f64280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 0x7f64280a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c031400 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 msgr2=0x7f643c1a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f643800b4a0 tx=0x7f643800b960 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 shutdown_connections 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f64280a3350 0x7f64280a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f643c10e500 0x7f643c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f643c101440 0x7f643c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 --2- 172.21.15.31:0/3509390250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643c100b20 0x7f643c1a0580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 >> 172.21.15.31:0/3509390250 conn(0x7f643c0fc7d0 msgr2=0x7f643c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 shutdown_connections 2024-09-15T23:58:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:40.732+0000 7f6443943640 1 -- 172.21.15.31:0/3509390250 wait complete. 2024-09-15T23:58:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:40 smithi031 ceph-mon[31203]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-15T23:58:41.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:40 smithi081 ceph-mon[35060]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-15T23:58:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:40 smithi102 ceph-mon[34458]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-09-15T23:58:41.465 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-15T23:58:39.192893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653774,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":306,"num_read_kb":276,"num_write":419,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347544,"kb_used_data":10776,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594280,"statfs":{"total":1151940427776,"available":1151584542720,"internally_reserved":0,"allocated":11034624,"data_stored":6955734,"data_compressed":112518,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":344634682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56134,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":144,"num_read_kb":137,"num_write":62,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002707"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154743+0000","last_change":"2024-09-15T23:58:11.160319+0000","last_active":"2024-09-15T23:58:12.154743+0000","last_peered":"2024-09-15T23:58:12.154743+0000","last_clean":"2024-09-15T23:58:12.154743+0000","last_became_active":"2024-09-15T23:58:11.159572+0000","last_became_peered":"2024-09-15T23:58:11.159572+0000","last_unstale":"2024-09-15T23:58:12.154743+0000","last_undegraded":"2024-09-15T23:58:12.154743+0000","last_fullsized":"2024-09-15T23:58:12.154743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:50.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150768+0000","last_change":"2024-09-15T23:58:11.152044+0000","last_active":"2024-09-15T23:58:12.150768+0000","last_peered":"2024-09-15T23:58:12.150768+0000","last_clean":"2024-09-15T23:58:12.150768+0000","last_became_active":"2024-09-15T23:58:11.151472+0000","last_became_peered":"2024-09-15T23:58:11.151472+0000","last_unstale":"2024-09-15T23:58:12.150768+0000","last_undegraded":"2024-09-15T23:58:12.150768+0000","last_fullsized":"2024-09-15T23:58:12.150768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:51.502165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150571+0000","last_change":"2024-09-15T23:58:11.151854+0000","last_active":"2024-09-15T23:58:12.150571+0000","last_peered":"2024-09-15T23:58:12.150571+0000","last_clean":"2024-09-15T23:58:12.150571+0000","last_became_active":"2024-09-15T23:58:11.151418+0000","last_became_peered":"2024-09-15T23:58:11.151418+0000","last_unstale":"2024-09-15T23:58:12.150571+0000","last_undegraded":"2024-09-15T23:58:12.150571+0000","last_fullsized":"2024-09-15T23:58:12.150571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:48.513653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149324+0000","last_change":"2024-09-15T23:58:11.162524+0000","last_active":"2024-09-15T23:58:12.149324+0000","last_peered":"2024-09-15T23:58:12.149324+0000","last_clean":"2024-09-15T23:58:12.149324+0000","last_became_active":"2024-09-15T23:58:11.161994+0000","last_became_peered":"2024-09-15T23:58:11.161994+0000","last_unstale":"2024-09-15T23:58:12.149324+0000","last_undegraded":"2024-09-15T23:58:12.149324+0000","last_fullsized":"2024-09-15T23:58:12.149324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:25.333515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150250+0000","last_change":"2024-09-15T23:58:11.143229+0000","last_active":"2024-09-15T23:58:12.150250+0000","last_peered":"2024-09-15T23:58:12.150250+0000","last_clean":"2024-09-15T23:58:12.150250+0000","last_became_active":"2024-09-15T23:58:11.142346+0000","last_became_peered":"2024-09-15T23:58:11.142346+0000","last_unstale":"2024-09-15T23:58:12.150250+0000","last_undegraded":"2024-09-15T23:58:12.150250+0000","last_fullsized":"2024-09-15T23:58:12.150250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:51.414916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136944+0000","last_change":"2024-09-15T23:58:11.152092+0000","last_active":"2024-09-15T23:58:12.136944+0000","last_peered":"2024-09-15T23:58:12.136944+0000","last_clean":"2024-09-15T23:58:12.136944+0000","last_became_active":"2024-09-15T23:58:11.151769+0000","last_became_peered":"2024-09-15T23:58:11.151769+0000","last_unstale":"2024-09-15T23:58:12.136944+0000","last_undegraded":"2024-09-15T23:58:12.136944+0000","last_fullsized":"2024-09-15T23:58:12.136944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:42.690545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150191+0000","last_change":"2024-09-15T23:58:11.158671+0000","last_active":"2024-09-15T23:58:12.150191+0000","last_peered":"2024-09-15T23:58:12.150191+0000","last_clean":"2024-09-15T23:58:12.150191+0000","last_became_active":"2024-09-15T23:58:11.158347+0000","last_became_peered":"2024-09-15T23:58:11.158347+0000","last_unstale":"2024-09-15T23:58:12.150191+0000","last_undegraded":"2024-09-15T23:58:12.150191+0000","last_fullsized":"2024-09-15T23:58:12.150191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:52.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136298+0000","last_change":"2024-09-15T23:58:11.149725+0000","last_active":"2024-09-15T23:58:12.136298+0000","last_peered":"2024-09-15T23:58:12.136298+0000","last_clean":"2024-09-15T23:58:12.136298+0000","last_became_active":"2024-09-15T23:58:11.149388+0000","last_became_peered":"2024-09-15T23:58:11.149388+0000","last_unstale":"2024-09-15T23:58:12.136298+0000","last_undegraded":"2024-09-15T23:58:12.136298+0000","last_fullsized":"2024-09-15T23:58:12.136298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:54.807443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149128+0000","last_change":"2024-09-15T23:58:11.155976+0000","last_active":"2024-09-15T23:58:12.149128+0000","last_peered":"2024-09-15T23:58:12.149128+0000","last_clean":"2024-09-15T23:58:12.149128+0000","last_became_active":"2024-09-15T23:58:11.155659+0000","last_became_peered":"2024-09-15T23:58:11.155659+0000","last_unstale":"2024-09-15T23:58:12.149128+0000","last_undegraded":"2024-09-15T23:58:12.149128+0000","last_fullsized":"2024-09-15T23:58:12.149128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:21.524908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137172+0000","last_change":"2024-09-15T23:58:11.157326+0000","last_active":"2024-09-15T23:58:12.137172+0000","last_peered":"2024-09-15T23:58:12.137172+0000","last_clean":"2024-09-15T23:58:12.137172+0000","last_became_active":"2024-09-15T23:58:11.156944+0000","last_became_peered":"2024-09-15T23:58:11.156944+0000","last_unstale":"2024-09-15T23:58:12.137172+0000","last_undegraded":"2024-09-15T23:58:12.137172+0000","last_fullsized":"2024-09-15T23:58:12.137172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:49.913437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151176+0000","last_change":"2024-09-15T23:58:11.154418+0000","last_active":"2024-09-15T23:58:12.151176+0000","last_peered":"2024-09-15T23:58:12.151176+0000","last_clean":"2024-09-15T23:58:12.151176+0000","last_became_active":"2024-09-15T23:58:11.154081+0000","last_became_peered":"2024-09-15T23:58:11.154081+0000","last_unstale":"2024-09-15T23:58:12.151176+0000","last_undegraded":"2024-09-15T23:58:12.151176+0000","last_fullsized":"2024-09-15T23:58:12.151176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:09.920736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149482+0000","last_change":"2024-09-15T23:58:11.158224+0000","last_active":"2024-09-15T23:58:12.149482+0000","last_peered":"2024-09-15T23:58:12.149482+0000","last_clean":"2024-09-15T23:58:12.149482+0000","last_became_active":"2024-09-15T23:58:11.156076+0000","last_became_peered":"2024-09-15T23:58:11.156076+0000","last_unstale":"2024-09-15T23:58:12.149482+0000","last_undegraded":"2024-09-15T23:58:12.149482+0000","last_fullsized":"2024-09-15T23:58:12.149482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:29.448669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148503+0000","last_change":"2024-09-15T23:58:11.151904+0000","last_active":"2024-09-15T23:58:12.148503+0000","last_peered":"2024-09-15T23:58:12.148503+0000","last_clean":"2024-09-15T23:58:12.148503+0000","last_became_active":"2024-09-15T23:58:11.151564+0000","last_became_peered":"2024-09-15T23:58:11.151564+0000","last_unstale":"2024-09-15T23:58:12.148503+0000","last_undegraded":"2024-09-15T23:58:12.148503+0000","last_fullsized":"2024-09-15T23:58:12.148503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:11.297350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151021+0000","last_change":"2024-09-15T23:58:11.148139+0000","last_active":"2024-09-15T23:58:12.151021+0000","last_peered":"2024-09-15T23:58:12.151021+0000","last_clean":"2024-09-15T23:58:12.151021+0000","last_became_active":"2024-09-15T23:58:11.147820+0000","last_became_peered":"2024-09-15T23:58:11.147820+0000","last_unstale":"2024-09-15T23:58:12.151021+0000","last_undegraded":"2024-09-15T23:58:12.151021+0000","last_fullsized":"2024-09-15T23:58:12.151021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:33.786315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150504+0000","last_change":"2024-09-15T23:58:11.160284+0000","last_active":"2024-09-15T23:58:12.150504+0000","last_peered":"2024-09-15T23:58:12.150504+0000","last_clean":"2024-09-15T23:58:12.150504+0000","last_became_active":"2024-09-15T23:58:11.159931+0000","last_became_peered":"2024-09-15T23:58:11.159931+0000","last_unstale":"2024-09-15T23:58:12.150504+0000","last_undegraded":"2024-09-15T23:58:12.150504+0000","last_fullsized":"2024-09-15T23:58:12.150504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:35.381030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147613+0000","last_change":"2024-09-15T23:58:11.162637+0000","last_active":"2024-09-15T23:58:12.147613+0000","last_peered":"2024-09-15T23:58:12.147613+0000","last_clean":"2024-09-15T23:58:12.147613+0000","last_became_active":"2024-09-15T23:58:11.162322+0000","last_became_peered":"2024-09-15T23:58:11.162322+0000","last_unstale":"2024-09-15T23:58:12.147613+0000","last_undegraded":"2024-09-15T23:58:12.147613+0000","last_fullsized":"2024-09-15T23:58:12.147613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:50.489012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154813+0000","last_change":"2024-09-15T23:58:11.160068+0000","last_active":"2024-09-15T23:58:12.154813+0000","last_peered":"2024-09-15T23:58:12.154813+0000","last_clean":"2024-09-15T23:58:12.154813+0000","last_became_active":"2024-09-15T23:58:11.159553+0000","last_became_peered":"2024-09-15T23:58:11.159553+0000","last_unstale":"2024-09-15T23:58:12.154813+0000","last_undegraded":"2024-09-15T23:58:12.154813+0000","last_fullsized":"2024-09-15T23:58:12.154813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:31.962049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147516+0000","last_change":"2024-09-15T23:58:11.148755+0000","last_active":"2024-09-15T23:58:12.147516+0000","last_peered":"2024-09-15T23:58:12.147516+0000","last_clean":"2024-09-15T23:58:12.147516+0000","last_became_active":"2024-09-15T23:58:11.148428+0000","last_became_peered":"2024-09-15T23:58:11.148428+0000","last_unstale":"2024-09-15T23:58:12.147516+0000","last_undegraded":"2024-09-15T23:58:12.147516+0000","last_fullsized":"2024-09-15T23:58:12.147516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.119847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148320+0000","last_change":"2024-09-15T23:58:11.143374+0000","last_active":"2024-09-15T23:58:12.148320+0000","last_peered":"2024-09-15T23:58:12.148320+0000","last_clean":"2024-09-15T23:58:12.148320+0000","last_became_active":"2024-09-15T23:58:11.143032+0000","last_became_peered":"2024-09-15T23:58:11.143032+0000","last_unstale":"2024-09-15T23:58:12.148320+0000","last_undegraded":"2024-09-15T23:58:12.148320+0000","last_fullsized":"2024-09-15T23:58:12.148320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:29.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153498+0000","last_change":"2024-09-15T23:58:11.160524+0000","last_active":"2024-09-15T23:58:12.153498+0000","last_peered":"2024-09-15T23:58:12.153498+0000","last_clean":"2024-09-15T23:58:12.153498+0000","last_became_active":"2024-09-15T23:58:11.160141+0000","last_became_peered":"2024-09-15T23:58:11.160141+0000","last_unstale":"2024-09-15T23:58:12.153498+0000","last_undegraded":"2024-09-15T23:58:12.153498+0000","last_fullsized":"2024-09-15T23:58:12.153498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:36.730764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148654+0000","last_change":"2024-09-15T23:58:11.153493+0000","last_active":"2024-09-15T23:58:12.148654+0000","last_peered":"2024-09-15T23:58:12.148654+0000","last_clean":"2024-09-15T23:58:12.148654+0000","last_became_active":"2024-09-15T23:58:11.153026+0000","last_became_peered":"2024-09-15T23:58:11.153026+0000","last_unstale":"2024-09-15T23:58:12.148654+0000","last_undegraded":"2024-09-15T23:58:12.148654+0000","last_fullsized":"2024-09-15T23:58:12.148654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:15.700186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148212+0000","last_change":"2024-09-15T23:58:11.159991+0000","last_active":"2024-09-15T23:58:12.148212+0000","last_peered":"2024-09-15T23:58:12.148212+0000","last_clean":"2024-09-15T23:58:12.148212+0000","last_became_active":"2024-09-15T23:58:11.159656+0000","last_became_peered":"2024-09-15T23:58:11.159656+0000","last_unstale":"2024-09-15T23:58:12.148212+0000","last_undegraded":"2024-09-15T23:58:12.148212+0000","last_fullsized":"2024-09-15T23:58:12.148212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:53.065297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136354+0000","last_change":"2024-09-15T23:58:11.154058+0000","last_active":"2024-09-15T23:58:12.136354+0000","last_peered":"2024-09-15T23:58:12.136354+0000","last_clean":"2024-09-15T23:58:12.136354+0000","last_became_active":"2024-09-15T23:58:11.153769+0000","last_became_peered":"2024-09-15T23:58:11.153769+0000","last_unstale":"2024-09-15T23:58:12.136354+0000","last_undegraded":"2024-09-15T23:58:12.136354+0000","last_fullsized":"2024-09-15T23:58:12.136354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:03:54.768080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136887+0000","last_change":"2024-09-15T23:58:11.148173+0000","last_active":"2024-09-15T23:58:12.136887+0000","last_peered":"2024-09-15T23:58:12.136887+0000","last_clean":"2024-09-15T23:58:12.136887+0000","last_became_active":"2024-09-15T23:58:11.147843+0000","last_became_peered":"2024-09-15T23:58:11.147843+0000","last_unstale":"2024-09-15T23:58:12.136887+0000","last_undegraded":"2024-09-15T23:58:12.136887+0000","last_fullsized":"2024-09-15T23:58:12.136887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:29.789888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136563+0000","last_change":"2024-09-15T23:58:11.162113+0000","last_active":"2024-09-15T23:58:12.136563+0000","last_peered":"2024-09-15T23:58:12.136563+0000","last_clean":"2024-09-15T23:58:12.136563+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.136563+0000","last_undegraded":"2024-09-15T23:58:12.136563+0000","last_fullsized":"2024-09-15T23:58:12.136563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:05.334727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148712+0000","last_change":"2024-09-15T23:58:11.150172+0000","last_active":"2024-09-15T23:58:12.148712+0000","last_peered":"2024-09-15T23:58:12.148712+0000","last_clean":"2024-09-15T23:58:12.148712+0000","last_became_active":"2024-09-15T23:58:11.149800+0000","last_became_peered":"2024-09-15T23:58:11.149800+0000","last_unstale":"2024-09-15T23:58:12.148712+0000","last_undegraded":"2024-09-15T23:58:12.148712+0000","last_fullsized":"2024-09-15T23:58:12.148712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:16.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154255+0000","last_change":"2024-09-15T23:58:11.153829+0000","last_active":"2024-09-15T23:58:12.154255+0000","last_peered":"2024-09-15T23:58:12.154255+0000","last_clean":"2024-09-15T23:58:12.154255+0000","last_became_active":"2024-09-15T23:58:11.153512+0000","last_became_peered":"2024-09-15T23:58:11.153512+0000","last_unstale":"2024-09-15T23:58:12.154255+0000","last_undegraded":"2024-09-15T23:58:12.154255+0000","last_fullsized":"2024-09-15T23:58:12.154255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:36.405204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147909+0000","last_change":"2024-09-15T23:58:11.162122+0000","last_active":"2024-09-15T23:58:12.147909+0000","last_peered":"2024-09-15T23:58:12.147909+0000","last_clean":"2024-09-15T23:58:12.147909+0000","last_became_active":"2024-09-15T23:58:11.161814+0000","last_became_peered":"2024-09-15T23:58:11.161814+0000","last_unstale":"2024-09-15T23:58:12.147909+0000","last_undegraded":"2024-09-15T23:58:12.147909+0000","last_fullsized":"2024-09-15T23:58:12.147909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:07.058458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150959+0000","last_change":"2024-09-15T23:58:11.157204+0000","last_active":"2024-09-15T23:58:12.150959+0000","last_peered":"2024-09-15T23:58:12.150959+0000","last_clean":"2024-09-15T23:58:12.150959+0000","last_became_active":"2024-09-15T23:58:11.156906+0000","last_became_peered":"2024-09-15T23:58:11.156906+0000","last_unstale":"2024-09-15T23:58:12.150959+0000","last_undegraded":"2024-09-15T23:58:12.150959+0000","last_fullsized":"2024-09-15T23:58:12.150959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:04.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148236+0000","last_change":"2024-09-15T23:58:11.150191+0000","last_active":"2024-09-15T23:58:12.148236+0000","last_peered":"2024-09-15T23:58:12.148236+0000","last_clean":"2024-09-15T23:58:12.148236+0000","last_became_active":"2024-09-15T23:58:11.149616+0000","last_became_peered":"2024-09-15T23:58:11.149616+0000","last_unstale":"2024-09-15T23:58:12.148236+0000","last_undegraded":"2024-09-15T23:58:12.148236+0000","last_fullsized":"2024-09-15T23:58:12.148236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:49.956626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148515+0000","last_change":"2024-09-15T23:58:11.162127+0000","last_active":"2024-09-15T23:58:12.148515+0000","last_peered":"2024-09-15T23:58:12.148515+0000","last_clean":"2024-09-15T23:58:12.148515+0000","last_became_active":"2024-09-15T23:58:11.161766+0000","last_became_peered":"2024-09-15T23:58:11.161766+0000","last_unstale":"2024-09-15T23:58:12.148515+0000","last_undegraded":"2024-09-15T23:58:12.148515+0000","last_fullsized":"2024-09-15T23:58:12.148515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:09.626747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154236+0000","last_change":"2024-09-15T23:58:11.153546+0000","last_active":"2024-09-15T23:58:12.154236+0000","last_peered":"2024-09-15T23:58:12.154236+0000","last_clean":"2024-09-15T23:58:12.154236+0000","last_became_active":"2024-09-15T23:58:11.153116+0000","last_became_peered":"2024-09-15T23:58:11.153116+0000","last_unstale":"2024-09-15T23:58:12.154236+0000","last_undegraded":"2024-09-15T23:58:12.154236+0000","last_fullsized":"2024-09-15T23:58:12.154236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:02.389068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148125+0000","last_change":"2024-09-15T23:58:11.147446+0000","last_active":"2024-09-15T23:58:12.148125+0000","last_peered":"2024-09-15T23:58:12.148125+0000","last_clean":"2024-09-15T23:58:12.148125+0000","last_became_active":"2024-09-15T23:58:11.146285+0000","last_became_peered":"2024-09-15T23:58:11.146285+0000","last_unstale":"2024-09-15T23:58:12.148125+0000","last_undegraded":"2024-09-15T23:58:12.148125+0000","last_fullsized":"2024-09-15T23:58:12.148125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:38.741814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:32.950663+0000","last_change":"2024-09-15T23:58:09.123719+0000","last_active":"2024-09-15T23:58:32.950663+0000","last_peered":"2024-09-15T23:58:32.950663+0000","last_clean":"2024-09-15T23:58:32.950663+0000","last_became_active":"2024-09-15T23:58:09.123082+0000","last_became_peered":"2024-09-15T23:58:09.123082+0000","last_unstale":"2024-09-15T23:58:32.950663+0000","last_undegraded":"2024-09-15T23:58:32.950663+0000","last_fullsized":"2024-09-15T23:58:32.950663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:46.704121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137195+0000","last_change":"2024-09-15T23:58:11.162571+0000","last_active":"2024-09-15T23:58:12.137195+0000","last_peered":"2024-09-15T23:58:12.137195+0000","last_clean":"2024-09-15T23:58:12.137195+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.137195+0000","last_undegraded":"2024-09-15T23:58:12.137195+0000","last_fullsized":"2024-09-15T23:58:12.137195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:58.134589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.290419+0000","last_change":"2024-09-15T23:58:09.125683+0000","last_active":"2024-09-15T23:58:22.290419+0000","last_peered":"2024-09-15T23:58:22.290419+0000","last_clean":"2024-09-15T23:58:22.290419+0000","last_became_active":"2024-09-15T23:58:09.125345+0000","last_became_peered":"2024-09-15T23:58:09.125345+0000","last_unstale":"2024-09-15T23:58:22.290419+0000","last_undegraded":"2024-09-15T23:58:22.290419+0000","last_fullsized":"2024-09-15T23:58:22.290419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:39.998850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284754+0000","last_change":"2024-09-15T23:58:09.116886+0000","last_active":"2024-09-15T23:58:21.284754+0000","last_peered":"2024-09-15T23:58:21.284754+0000","last_clean":"2024-09-15T23:58:21.284754+0000","last_became_active":"2024-09-15T23:58:09.116510+0000","last_became_peered":"2024-09-15T23:58:09.116510+0000","last_unstale":"2024-09-15T23:58:21.284754+0000","last_undegraded":"2024-09-15T23:58:21.284754+0000","last_fullsized":"2024-09-15T23:58:21.284754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:10.368106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136363+0000","last_change":"2024-09-15T23:58:11.144759+0000","last_active":"2024-09-15T23:58:12.136363+0000","last_peered":"2024-09-15T23:58:12.136363+0000","last_clean":"2024-09-15T23:58:12.136363+0000","last_became_active":"2024-09-15T23:58:11.144435+0000","last_became_peered":"2024-09-15T23:58:11.144435+0000","last_unstale":"2024-09-15T23:58:12.136363+0000","last_undegraded":"2024-09-15T23:58:12.136363+0000","last_fullsized":"2024-09-15T23:58:12.136363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:32.787727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300459+0000","last_change":"2024-09-15T23:58:09.124938+0000","last_active":"2024-09-15T23:58:23.300459+0000","last_peered":"2024-09-15T23:58:23.300459+0000","last_clean":"2024-09-15T23:58:23.300459+0000","last_became_active":"2024-09-15T23:58:09.124532+0000","last_became_peered":"2024-09-15T23:58:09.124532+0000","last_unstale":"2024-09-15T23:58:23.300459+0000","last_undegraded":"2024-09-15T23:58:23.300459+0000","last_fullsized":"2024-09-15T23:58:23.300459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148543+0000","last_change":"2024-09-15T23:58:11.145940+0000","last_active":"2024-09-15T23:58:12.148543+0000","last_peered":"2024-09-15T23:58:12.148543+0000","last_clean":"2024-09-15T23:58:12.148543+0000","last_became_active":"2024-09-15T23:58:11.145642+0000","last_became_peered":"2024-09-15T23:58:11.145642+0000","last_unstale":"2024-09-15T23:58:12.148543+0000","last_undegraded":"2024-09-15T23:58:12.148543+0000","last_fullsized":"2024-09-15T23:58:12.148543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:58.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:36.955033+0000","last_change":"2024-09-15T23:58:09.126742+0000","last_active":"2024-09-15T23:58:36.955033+0000","last_peered":"2024-09-15T23:58:36.955033+0000","last_clean":"2024-09-15T23:58:36.955033+0000","last_became_active":"2024-09-15T23:58:09.126432+0000","last_became_peered":"2024-09-15T23:58:09.126432+0000","last_unstale":"2024-09-15T23:58:36.955033+0000","last_undegraded":"2024-09-15T23:58:36.955033+0000","last_fullsized":"2024-09-15T23:58:36.955033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:11.582346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148667+0000","last_change":"2024-09-15T23:58:11.150130+0000","last_active":"2024-09-15T23:58:12.148667+0000","last_peered":"2024-09-15T23:58:12.148667+0000","last_clean":"2024-09-15T23:58:12.148667+0000","last_became_active":"2024-09-15T23:58:11.149531+0000","last_became_peered":"2024-09-15T23:58:11.149531+0000","last_unstale":"2024-09-15T23:58:12.148667+0000","last_undegraded":"2024-09-15T23:58:12.148667+0000","last_fullsized":"2024-09-15T23:58:12.148667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:07.206505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299397+0000","last_change":"2024-09-15T23:58:09.121229+0000","last_active":"2024-09-15T23:58:23.299397+0000","last_peered":"2024-09-15T23:58:23.299397+0000","last_clean":"2024-09-15T23:58:23.299397+0000","last_became_active":"2024-09-15T23:58:09.120808+0000","last_became_peered":"2024-09-15T23:58:09.120808+0000","last_unstale":"2024-09-15T23:58:23.299397+0000","last_undegraded":"2024-09-15T23:58:23.299397+0000","last_fullsized":"2024-09-15T23:58:23.299397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:34.155774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151252+0000","last_change":"2024-09-15T23:58:11.157253+0000","last_active":"2024-09-15T23:58:12.151252+0000","last_peered":"2024-09-15T23:58:12.151252+0000","last_clean":"2024-09-15T23:58:12.151252+0000","last_became_active":"2024-09-15T23:58:11.156952+0000","last_became_peered":"2024-09-15T23:58:11.156952+0000","last_unstale":"2024-09-15T23:58:12.151252+0000","last_undegraded":"2024-09-15T23:58:12.151252+0000","last_fullsized":"2024-09-15T23:58:12.151252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:09.840371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289729+0000","last_change":"2024-09-15T23:58:09.322776+0000","last_active":"2024-09-15T23:58:22.289729+0000","last_peered":"2024-09-15T23:58:22.289729+0000","last_clean":"2024-09-15T23:58:22.289729+0000","last_became_active":"2024-09-15T23:58:09.322304+0000","last_became_peered":"2024-09-15T23:58:09.322304+0000","last_unstale":"2024-09-15T23:58:22.289729+0000","last_undegraded":"2024-09-15T23:58:22.289729+0000","last_fullsized":"2024-09-15T23:58:22.289729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:51.413578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148251+0000","last_change":"2024-09-15T23:58:11.165067+0000","last_active":"2024-09-15T23:58:12.148251+0000","last_peered":"2024-09-15T23:58:12.148251+0000","last_clean":"2024-09-15T23:58:12.148251+0000","last_became_active":"2024-09-15T23:58:11.163722+0000","last_became_peered":"2024-09-15T23:58:11.163722+0000","last_unstale":"2024-09-15T23:58:12.148251+0000","last_undegraded":"2024-09-15T23:58:12.148251+0000","last_fullsized":"2024-09-15T23:58:12.148251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:01.354822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'28","reported_seq":80,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:36.955481+0000","last_change":"2024-09-15T23:58:09.128077+0000","last_active":"2024-09-15T23:58:36.955481+0000","last_peered":"2024-09-15T23:58:36.955481+0000","last_clean":"2024-09-15T23:58:36.955481+0000","last_became_active":"2024-09-15T23:58:09.127215+0000","last_became_peered":"2024-09-15T23:58:09.127215+0000","last_unstale":"2024-09-15T23:58:36.955481+0000","last_undegraded":"2024-09-15T23:58:36.955481+0000","last_fullsized":"2024-09-15T23:58:36.955481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:47.875279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56357,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":71,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150490+0000","last_change":"2024-09-15T23:58:11.156020+0000","last_active":"2024-09-15T23:58:12.150490+0000","last_peered":"2024-09-15T23:58:12.150490+0000","last_clean":"2024-09-15T23:58:12.150490+0000","last_became_active":"2024-09-15T23:58:11.155634+0000","last_became_peered":"2024-09-15T23:58:11.155634+0000","last_unstale":"2024-09-15T23:58:12.150490+0000","last_undegraded":"2024-09-15T23:58:12.150490+0000","last_fullsized":"2024-09-15T23:58:12.150490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934453+0000","last_change":"2024-09-15T23:58:09.123084+0000","last_active":"2024-09-15T23:58:23.934453+0000","last_peered":"2024-09-15T23:58:23.934453+0000","last_clean":"2024-09-15T23:58:23.934453+0000","last_became_active":"2024-09-15T23:58:09.122594+0000","last_became_peered":"2024-09-15T23:58:09.122594+0000","last_unstale":"2024-09-15T23:58:23.934453+0000","last_undegraded":"2024-09-15T23:58:23.934453+0000","last_fullsized":"2024-09-15T23:58:23.934453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:47.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149295+0000","last_change":"2024-09-15T23:58:11.155082+0000","last_active":"2024-09-15T23:58:12.149295+0000","last_peered":"2024-09-15T23:58:12.149295+0000","last_clean":"2024-09-15T23:58:12.149295+0000","last_became_active":"2024-09-15T23:58:11.154764+0000","last_became_peered":"2024-09-15T23:58:11.154764+0000","last_unstale":"2024-09-15T23:58:12.149295+0000","last_undegraded":"2024-09-15T23:58:12.149295+0000","last_fullsized":"2024-09-15T23:58:12.149295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:47.170383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300267+0000","last_change":"2024-09-15T23:58:09.124416+0000","last_active":"2024-09-15T23:58:23.300267+0000","last_peered":"2024-09-15T23:58:23.300267+0000","last_clean":"2024-09-15T23:58:23.300267+0000","last_became_active":"2024-09-15T23:58:09.123836+0000","last_became_peered":"2024-09-15T23:58:09.123836+0000","last_unstale":"2024-09-15T23:58:23.300267+0000","last_undegraded":"2024-09-15T23:58:23.300267+0000","last_fullsized":"2024-09-15T23:58:23.300267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:35.730677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150934+0000","last_change":"2024-09-15T23:58:11.162220+0000","last_active":"2024-09-15T23:58:12.150934+0000","last_peered":"2024-09-15T23:58:12.150934+0000","last_clean":"2024-09-15T23:58:12.150934+0000","last_became_active":"2024-09-15T23:58:11.161887+0000","last_became_peered":"2024-09-15T23:58:11.161887+0000","last_unstale":"2024-09-15T23:58:12.150934+0000","last_undegraded":"2024-09-15T23:58:12.150934+0000","last_fullsized":"2024-09-15T23:58:12.150934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:35.070135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:33.951146+0000","last_change":"2024-09-15T23:58:09.123010+0000","last_active":"2024-09-15T23:58:33.951146+0000","last_peered":"2024-09-15T23:58:33.951146+0000","last_clean":"2024-09-15T23:58:33.951146+0000","last_became_active":"2024-09-15T23:58:09.122615+0000","last_became_peered":"2024-09-15T23:58:09.122615+0000","last_unstale":"2024-09-15T23:58:33.951146+0000","last_undegraded":"2024-09-15T23:58:33.951146+0000","last_fullsized":"2024-09-15T23:58:33.951146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:09.634409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150261+0000","last_change":"2024-09-15T23:58:11.159069+0000","last_active":"2024-09-15T23:58:12.150261+0000","last_peered":"2024-09-15T23:58:12.150261+0000","last_clean":"2024-09-15T23:58:12.150261+0000","last_became_active":"2024-09-15T23:58:11.158659+0000","last_became_peered":"2024-09-15T23:58:11.158659+0000","last_unstale":"2024-09-15T23:58:12.150261+0000","last_undegraded":"2024-09-15T23:58:12.150261+0000","last_fullsized":"2024-09-15T23:58:12.150261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:24.336089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298401+0000","last_change":"2024-09-15T23:58:09.322903+0000","last_active":"2024-09-15T23:58:23.298401+0000","last_peered":"2024-09-15T23:58:23.298401+0000","last_clean":"2024-09-15T23:58:23.298401+0000","last_became_active":"2024-09-15T23:58:09.322326+0000","last_became_peered":"2024-09-15T23:58:09.322326+0000","last_unstale":"2024-09-15T23:58:23.298401+0000","last_undegraded":"2024-09-15T23:58:23.298401+0000","last_fullsized":"2024-09-15T23:58:23.298401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:58.749293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136919+0000","last_change":"2024-09-15T23:58:11.162104+0000","last_active":"2024-09-15T23:58:12.136919+0000","last_peered":"2024-09-15T23:58:12.136919+0000","last_clean":"2024-09-15T23:58:12.136919+0000","last_became_active":"2024-09-15T23:58:11.161739+0000","last_became_peered":"2024-09-15T23:58:11.161739+0000","last_unstale":"2024-09-15T23:58:12.136919+0000","last_undegraded":"2024-09-15T23:58:12.136919+0000","last_fullsized":"2024-09-15T23:58:12.136919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:55.316245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.279815+0000","last_change":"2024-09-15T23:58:09.119975+0000","last_active":"2024-09-15T23:58:22.279815+0000","last_peered":"2024-09-15T23:58:22.279815+0000","last_clean":"2024-09-15T23:58:22.279815+0000","last_became_active":"2024-09-15T23:58:09.119340+0000","last_became_peered":"2024-09-15T23:58:09.119340+0000","last_unstale":"2024-09-15T23:58:22.279815+0000","last_undegraded":"2024-09-15T23:58:22.279815+0000","last_fullsized":"2024-09-15T23:58:22.279815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:22:14.858076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153874+0000","last_change":"2024-09-15T23:58:11.157389+0000","last_active":"2024-09-15T23:58:12.153874+0000","last_peered":"2024-09-15T23:58:12.153874+0000","last_clean":"2024-09-15T23:58:12.153874+0000","last_became_active":"2024-09-15T23:58:11.157033+0000","last_became_peered":"2024-09-15T23:58:11.157033+0000","last_unstale":"2024-09-15T23:58:12.153874+0000","last_undegraded":"2024-09-15T23:58:12.153874+0000","last_fullsized":"2024-09-15T23:58:12.153874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:49.149860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299541+0000","last_change":"2024-09-15T23:58:09.120937+0000","last_active":"2024-09-15T23:58:23.299541+0000","last_peered":"2024-09-15T23:58:23.299541+0000","last_clean":"2024-09-15T23:58:23.299541+0000","last_became_active":"2024-09-15T23:58:09.120586+0000","last_became_peered":"2024-09-15T23:58:09.120586+0000","last_unstale":"2024-09-15T23:58:23.299541+0000","last_undegraded":"2024-09-15T23:58:23.299541+0000","last_fullsized":"2024-09-15T23:58:23.299541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:18.432631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150943+0000","last_change":"2024-09-15T23:58:11.155316+0000","last_active":"2024-09-15T23:58:12.150943+0000","last_peered":"2024-09-15T23:58:12.150943+0000","last_clean":"2024-09-15T23:58:12.150943+0000","last_became_active":"2024-09-15T23:58:11.154975+0000","last_became_peered":"2024-09-15T23:58:11.154975+0000","last_unstale":"2024-09-15T23:58:12.150943+0000","last_undegraded":"2024-09-15T23:58:12.150943+0000","last_fullsized":"2024-09-15T23:58:12.150943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:09.856126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300569+0000","last_change":"2024-09-15T23:58:09.125045+0000","last_active":"2024-09-15T23:58:23.300569+0000","last_peered":"2024-09-15T23:58:23.300569+0000","last_clean":"2024-09-15T23:58:23.300569+0000","last_became_active":"2024-09-15T23:58:09.124577+0000","last_became_peered":"2024-09-15T23:58:09.124577+0000","last_unstale":"2024-09-15T23:58:23.300569+0000","last_undegraded":"2024-09-15T23:58:23.300569+0000","last_fullsized":"2024-09-15T23:58:23.300569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:48.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150843+0000","last_change":"2024-09-15T23:58:11.160241+0000","last_active":"2024-09-15T23:58:12.150843+0000","last_peered":"2024-09-15T23:58:12.150843+0000","last_clean":"2024-09-15T23:58:12.150843+0000","last_became_active":"2024-09-15T23:58:11.159886+0000","last_became_peered":"2024-09-15T23:58:11.159886+0000","last_unstale":"2024-09-15T23:58:12.150843+0000","last_undegraded":"2024-09-15T23:58:12.150843+0000","last_fullsized":"2024-09-15T23:58:12.150843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:02.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:35.954338+0000","last_change":"2024-09-15T23:58:09.482888+0000","last_active":"2024-09-15T23:58:35.954338+0000","last_peered":"2024-09-15T23:58:35.954338+0000","last_clean":"2024-09-15T23:58:35.954338+0000","last_became_active":"2024-09-15T23:58:09.482497+0000","last_became_peered":"2024-09-15T23:58:09.482497+0000","last_unstale":"2024-09-15T23:58:35.954338+0000","last_undegraded":"2024-09-15T23:58:35.954338+0000","last_fullsized":"2024-09-15T23:58:35.954338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:59.320794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151266+0000","last_change":"2024-09-15T23:58:11.159452+0000","last_active":"2024-09-15T23:58:12.151266+0000","last_peered":"2024-09-15T23:58:12.151266+0000","last_clean":"2024-09-15T23:58:12.151266+0000","last_became_active":"2024-09-15T23:58:11.157610+0000","last_became_peered":"2024-09-15T23:58:11.157610+0000","last_unstale":"2024-09-15T23:58:12.151266+0000","last_undegraded":"2024-09-15T23:58:12.151266+0000","last_fullsized":"2024-09-15T23:58:12.151266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:17.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150373+0000","last_change":"2024-09-15T23:56:56.876989+0000","last_active":"2024-09-15T23:58:12.150373+0000","last_peered":"2024-09-15T23:58:12.150373+0000","last_clean":"2024-09-15T23:58:12.150373+0000","last_became_active":"2024-09-15T23:56:56.876413+0000","last_became_peered":"2024-09-15T23:56:56.876413+0000","last_unstale":"2024-09-15T23:58:12.150373+0000","last_undegraded":"2024-09-15T23:58:12.150373+0000","last_fullsized":"2024-09-15T23:58:12.150373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_clean_scrub_stamp":"2024-09-15T23:53:27.674229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:06.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934257+0000","last_change":"2024-09-15T23:58:09.124514+0000","last_active":"2024-09-15T23:58:23.934257+0000","last_peered":"2024-09-15T23:58:23.934257+0000","last_clean":"2024-09-15T23:58:23.934257+0000","last_became_active":"2024-09-15T23:58:09.122124+0000","last_became_peered":"2024-09-15T23:58:09.122124+0000","last_unstale":"2024-09-15T23:58:23.934257+0000","last_undegraded":"2024-09-15T23:58:23.934257+0000","last_fullsized":"2024-09-15T23:58:23.934257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:13.563073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136568+0000","last_change":"2024-09-15T23:58:11.149559+0000","last_active":"2024-09-15T23:58:12.136568+0000","last_peered":"2024-09-15T23:58:12.136568+0000","last_clean":"2024-09-15T23:58:12.136568+0000","last_became_active":"2024-09-15T23:58:11.149215+0000","last_became_peered":"2024-09-15T23:58:11.149215+0000","last_unstale":"2024-09-15T23:58:12.136568+0000","last_undegraded":"2024-09-15T23:58:12.136568+0000","last_fullsized":"2024-09-15T23:58:12.136568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:16.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284817+0000","last_change":"2024-09-15T23:58:09.127770+0000","last_active":"2024-09-15T23:58:21.284817+0000","last_peered":"2024-09-15T23:58:21.284817+0000","last_clean":"2024-09-15T23:58:21.284817+0000","last_became_active":"2024-09-15T23:58:09.127404+0000","last_became_peered":"2024-09-15T23:58:09.127404+0000","last_unstale":"2024-09-15T23:58:21.284817+0000","last_undegraded":"2024-09-15T23:58:21.284817+0000","last_fullsized":"2024-09-15T23:58:21.284817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:48.176290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148872+0000","last_change":"2024-09-15T23:58:11.162161+0000","last_active":"2024-09-15T23:58:12.148872+0000","last_peered":"2024-09-15T23:58:12.148872+0000","last_clean":"2024-09-15T23:58:12.148872+0000","last_became_active":"2024-09-15T23:58:11.161879+0000","last_became_peered":"2024-09-15T23:58:11.161879+0000","last_unstale":"2024-09-15T23:58:12.148872+0000","last_undegraded":"2024-09-15T23:58:12.148872+0000","last_fullsized":"2024-09-15T23:58:12.148872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:34.951818+0000","last_change":"2024-09-15T23:58:09.322611+0000","last_active":"2024-09-15T23:58:34.951818+0000","last_peered":"2024-09-15T23:58:34.951818+0000","last_clean":"2024-09-15T23:58:34.951818+0000","last_became_active":"2024-09-15T23:58:09.322254+0000","last_became_peered":"2024-09-15T23:58:09.322254+0000","last_unstale":"2024-09-15T23:58:34.951818+0000","last_undegraded":"2024-09-15T23:58:34.951818+0000","last_fullsized":"2024-09-15T23:58:34.951818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:03.803927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137151+0000","last_change":"2024-09-15T23:58:11.152219+0000","last_active":"2024-09-15T23:58:12.137151+0000","last_peered":"2024-09-15T23:58:12.137151+0000","last_clean":"2024-09-15T23:58:12.137151+0000","last_became_active":"2024-09-15T23:58:11.151885+0000","last_became_peered":"2024-09-15T23:58:11.151885+0000","last_unstale":"2024-09-15T23:58:12.137151+0000","last_undegraded":"2024-09-15T23:58:12.137151+0000","last_fullsized":"2024-09-15T23:58:12.137151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:10.780163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:34.952053+0000","last_change":"2024-09-15T23:58:09.125152+0000","last_active":"2024-09-15T23:58:34.952053+0000","last_peered":"2024-09-15T23:58:34.952053+0000","last_clean":"2024-09-15T23:58:34.952053+0000","last_became_active":"2024-09-15T23:58:09.124754+0000","last_became_peered":"2024-09-15T23:58:09.124754+0000","last_unstale":"2024-09-15T23:58:34.952053+0000","last_undegraded":"2024-09-15T23:58:34.952053+0000","last_fullsized":"2024-09-15T23:58:34.952053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:02.094909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136921+0000","last_change":"2024-09-15T23:58:11.160257+0000","last_active":"2024-09-15T23:58:12.136921+0000","last_peered":"2024-09-15T23:58:12.136921+0000","last_clean":"2024-09-15T23:58:12.136921+0000","last_became_active":"2024-09-15T23:58:11.159868+0000","last_became_peered":"2024-09-15T23:58:11.159868+0000","last_unstale":"2024-09-15T23:58:12.136921+0000","last_undegraded":"2024-09-15T23:58:12.136921+0000","last_fullsized":"2024-09-15T23:58:12.136921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.528002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.226786+0000","last_change":"2024-09-15T23:58:09.323034+0000","last_active":"2024-09-15T23:58:12.226786+0000","last_peered":"2024-09-15T23:58:12.226786+0000","last_clean":"2024-09-15T23:58:12.226786+0000","last_became_active":"2024-09-15T23:58:09.322658+0000","last_became_peered":"2024-09-15T23:58:09.322658+0000","last_unstale":"2024-09-15T23:58:12.226786+0000","last_undegraded":"2024-09-15T23:58:12.226786+0000","last_fullsized":"2024-09-15T23:58:12.226786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:20:11.175093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.862670+0000","last_change":"2024-09-15T23:58:11.149216+0000","last_active":"2024-09-15T23:58:12.862670+0000","last_peered":"2024-09-15T23:58:12.862670+0000","last_clean":"2024-09-15T23:58:12.862670+0000","last_became_active":"2024-09-15T23:58:11.148893+0000","last_became_peered":"2024-09-15T23:58:11.148893+0000","last_unstale":"2024-09-15T23:58:12.862670+0000","last_undegraded":"2024-09-15T23:58:12.862670+0000","last_fullsized":"2024-09-15T23:58:12.862670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:07.749101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300019+0000","last_change":"2024-09-15T23:58:09.323030+0000","last_active":"2024-09-15T23:58:23.300019+0000","last_peered":"2024-09-15T23:58:23.300019+0000","last_clean":"2024-09-15T23:58:23.300019+0000","last_became_active":"2024-09-15T23:58:09.322569+0000","last_became_peered":"2024-09-15T23:58:09.322569+0000","last_unstale":"2024-09-15T23:58:23.300019+0000","last_undegraded":"2024-09-15T23:58:23.300019+0000","last_fullsized":"2024-09-15T23:58:23.300019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:52.811395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148063+0000","last_change":"2024-09-15T23:58:11.154084+0000","last_active":"2024-09-15T23:58:12.148063+0000","last_peered":"2024-09-15T23:58:12.148063+0000","last_clean":"2024-09-15T23:58:12.148063+0000","last_became_active":"2024-09-15T23:58:11.153720+0000","last_became_peered":"2024-09-15T23:58:11.153720+0000","last_unstale":"2024-09-15T23:58:12.148063+0000","last_undegraded":"2024-09-15T23:58:12.148063+0000","last_fullsized":"2024-09-15T23:58:12.148063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:07.468812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:36.149861+0000","last_change":"2024-09-15T23:58:09.116996+0000","last_active":"2024-09-15T23:58:36.149861+0000","last_peered":"2024-09-15T23:58:36.149861+0000","last_clean":"2024-09-15T23:58:36.149861+0000","last_became_active":"2024-09-15T23:58:09.116531+0000","last_became_peered":"2024-09-15T23:58:09.116531+0000","last_unstale":"2024-09-15T23:58:36.149861+0000","last_undegraded":"2024-09-15T23:58:36.149861+0000","last_fullsized":"2024-09-15T23:58:36.149861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.094719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136595+0000","last_change":"2024-09-15T23:58:11.155704+0000","last_active":"2024-09-15T23:58:12.136595+0000","last_peered":"2024-09-15T23:58:12.136595+0000","last_clean":"2024-09-15T23:58:12.136595+0000","last_became_active":"2024-09-15T23:58:11.155344+0000","last_became_peered":"2024-09-15T23:58:11.155344+0000","last_unstale":"2024-09-15T23:58:12.136595+0000","last_undegraded":"2024-09-15T23:58:12.136595+0000","last_fullsized":"2024-09-15T23:58:12.136595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:50.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298948+0000","last_change":"2024-09-15T23:58:09.482837+0000","last_active":"2024-09-15T23:58:23.298948+0000","last_peered":"2024-09-15T23:58:23.298948+0000","last_clean":"2024-09-15T23:58:23.298948+0000","last_became_active":"2024-09-15T23:58:09.482444+0000","last_became_peered":"2024-09-15T23:58:09.482444+0000","last_unstale":"2024-09-15T23:58:23.298948+0000","last_undegraded":"2024-09-15T23:58:23.298948+0000","last_fullsized":"2024-09-15T23:58:23.298948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:05.962471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150939+0000","last_change":"2024-09-15T23:58:11.157249+0000","last_active":"2024-09-15T23:58:12.150939+0000","last_peered":"2024-09-15T23:58:12.150939+0000","last_clean":"2024-09-15T23:58:12.150939+0000","last_became_active":"2024-09-15T23:58:11.156871+0000","last_became_peered":"2024-09-15T23:58:11.156871+0000","last_unstale":"2024-09-15T23:58:12.150939+0000","last_undegraded":"2024-09-15T23:58:12.150939+0000","last_fullsized":"2024-09-15T23:58:12.150939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:16.986165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279567+0000","last_change":"2024-09-15T23:58:09.324320+0000","last_active":"2024-09-15T23:58:20.279567+0000","last_peered":"2024-09-15T23:58:20.279567+0000","last_clean":"2024-09-15T23:58:20.279567+0000","last_became_active":"2024-09-15T23:58:09.323996+0000","last_became_peered":"2024-09-15T23:58:09.323996+0000","last_unstale":"2024-09-15T23:58:20.279567+0000","last_undegraded":"2024-09-15T23:58:20.279567+0000","last_fullsized":"2024-09-15T23:58:20.279567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:34.456229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147904+0000","last_change":"2024-09-15T23:58:11.146836+0000","last_active":"2024-09-15T23:58:12.147904+0000","last_peered":"2024-09-15T23:58:12.147904+0000","last_clean":"2024-09-15T23:58:12.147904+0000","last_became_active":"2024-09-15T23:58:11.146470+0000","last_became_peered":"2024-09-15T23:58:11.146470+0000","last_unstale":"2024-09-15T23:58:12.147904+0000","last_undegraded":"2024-09-15T23:58:12.147904+0000","last_fullsized":"2024-09-15T23:58:12.147904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.721954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300504+0000","last_change":"2024-09-15T23:58:09.121920+0000","last_active":"2024-09-15T23:58:23.300504+0000","last_peered":"2024-09-15T23:58:23.300504+0000","last_clean":"2024-09-15T23:58:23.300504+0000","last_became_active":"2024-09-15T23:58:09.121605+0000","last_became_peered":"2024-09-15T23:58:09.121605+0000","last_unstale":"2024-09-15T23:58:23.300504+0000","last_undegraded":"2024-09-15T23:58:23.300504+0000","last_fullsized":"2024-09-15T23:58:23.300504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:12.707690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154389+0000","last_change":"2024-09-15T23:58:11.153676+0000","last_active":"2024-09-15T23:58:12.154389+0000","last_peered":"2024-09-15T23:58:12.154389+0000","last_clean":"2024-09-15T23:58:12.154389+0000","last_became_active":"2024-09-15T23:58:11.152866+0000","last_became_peered":"2024-09-15T23:58:11.152866+0000","last_unstale":"2024-09-15T23:58:12.154389+0000","last_undegraded":"2024-09-15T23:58:12.154389+0000","last_fullsized":"2024-09-15T23:58:12.154389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:02.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289656+0000","last_change":"2024-09-15T23:58:09.121440+0000","last_active":"2024-09-15T23:58:22.289656+0000","last_peered":"2024-09-15T23:58:22.289656+0000","last_clean":"2024-09-15T23:58:22.289656+0000","last_became_active":"2024-09-15T23:58:09.121006+0000","last_became_peered":"2024-09-15T23:58:09.121006+0000","last_unstale":"2024-09-15T23:58:22.289656+0000","last_undegraded":"2024-09-15T23:58:22.289656+0000","last_fullsized":"2024-09-15T23:58:22.289656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:32.584222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136632+0000","last_change":"2024-09-15T23:58:11.155311+0000","last_active":"2024-09-15T23:58:12.136632+0000","last_peered":"2024-09-15T23:58:12.136632+0000","last_clean":"2024-09-15T23:58:12.136632+0000","last_became_active":"2024-09-15T23:58:11.154954+0000","last_became_peered":"2024-09-15T23:58:11.154954+0000","last_unstale":"2024-09-15T23:58:12.136632+0000","last_undegraded":"2024-09-15T23:58:12.136632+0000","last_fullsized":"2024-09-15T23:58:12.136632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:52.131451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279891+0000","last_change":"2024-09-15T23:58:09.126143+0000","last_active":"2024-09-15T23:58:20.279891+0000","last_peered":"2024-09-15T23:58:20.279891+0000","last_clean":"2024-09-15T23:58:20.279891+0000","last_became_active":"2024-09-15T23:58:09.125785+0000","last_became_peered":"2024-09-15T23:58:09.125785+0000","last_unstale":"2024-09-15T23:58:20.279891+0000","last_undegraded":"2024-09-15T23:58:20.279891+0000","last_fullsized":"2024-09-15T23:58:20.279891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:22.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150690+0000","last_change":"2024-09-15T23:58:11.160834+0000","last_active":"2024-09-15T23:58:12.150690+0000","last_peered":"2024-09-15T23:58:12.150690+0000","last_clean":"2024-09-15T23:58:12.150690+0000","last_became_active":"2024-09-15T23:58:11.160458+0000","last_became_peered":"2024-09-15T23:58:11.160458+0000","last_unstale":"2024-09-15T23:58:12.150690+0000","last_undegraded":"2024-09-15T23:58:12.150690+0000","last_fullsized":"2024-09-15T23:58:12.150690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:36.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137347+0000","last_change":"2024-09-15T23:58:11.151745+0000","last_active":"2024-09-15T23:58:12.137347+0000","last_peered":"2024-09-15T23:58:12.137347+0000","last_clean":"2024-09-15T23:58:12.137347+0000","last_became_active":"2024-09-15T23:58:11.150864+0000","last_became_peered":"2024-09-15T23:58:11.150864+0000","last_unstale":"2024-09-15T23:58:12.137347+0000","last_undegraded":"2024-09-15T23:58:12.137347+0000","last_fullsized":"2024-09-15T23:58:12.137347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:34.578125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300990+0000","last_change":"2024-09-15T23:58:09.121095+0000","last_active":"2024-09-15T23:58:23.300990+0000","last_peered":"2024-09-15T23:58:23.300990+0000","last_clean":"2024-09-15T23:58:23.300990+0000","last_became_active":"2024-09-15T23:58:09.120671+0000","last_became_peered":"2024-09-15T23:58:09.120671+0000","last_unstale":"2024-09-15T23:58:23.300990+0000","last_undegraded":"2024-09-15T23:58:23.300990+0000","last_fullsized":"2024-09-15T23:58:23.300990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:32.099810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148041+0000","last_change":"2024-09-15T23:58:11.165256+0000","last_active":"2024-09-15T23:58:12.148041+0000","last_peered":"2024-09-15T23:58:12.148041+0000","last_clean":"2024-09-15T23:58:12.148041+0000","last_became_active":"2024-09-15T23:58:11.163663+0000","last_became_peered":"2024-09-15T23:58:11.163663+0000","last_unstale":"2024-09-15T23:58:12.148041+0000","last_undegraded":"2024-09-15T23:58:12.148041+0000","last_fullsized":"2024-09-15T23:58:12.148041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:02.112109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934080+0000","last_change":"2024-09-15T23:58:09.125833+0000","last_active":"2024-09-15T23:58:23.934080+0000","last_peered":"2024-09-15T23:58:23.934080+0000","last_clean":"2024-09-15T23:58:23.934080+0000","last_became_active":"2024-09-15T23:58:09.125411+0000","last_became_peered":"2024-09-15T23:58:09.125411+0000","last_unstale":"2024-09-15T23:58:23.934080+0000","last_undegraded":"2024-09-15T23:58:23.934080+0000","last_fullsized":"2024-09-15T23:58:23.934080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:17.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151115+0000","last_change":"2024-09-15T23:58:11.159923+0000","last_active":"2024-09-15T23:58:12.151115+0000","last_peered":"2024-09-15T23:58:12.151115+0000","last_clean":"2024-09-15T23:58:12.151115+0000","last_became_active":"2024-09-15T23:58:11.159489+0000","last_became_peered":"2024-09-15T23:58:11.159489+0000","last_unstale":"2024-09-15T23:58:12.151115+0000","last_undegraded":"2024-09-15T23:58:12.151115+0000","last_fullsized":"2024-09-15T23:58:12.151115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:30.864728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153095+0000","last_change":"2024-09-15T23:58:09.322767+0000","last_active":"2024-09-15T23:58:12.153095+0000","last_peered":"2024-09-15T23:58:12.153095+0000","last_clean":"2024-09-15T23:58:12.153095+0000","last_became_active":"2024-09-15T23:58:09.322289+0000","last_became_peered":"2024-09-15T23:58:09.322289+0000","last_unstale":"2024-09-15T23:58:12.153095+0000","last_undegraded":"2024-09-15T23:58:12.153095+0000","last_fullsized":"2024-09-15T23:58:12.153095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:12.168772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136810+0000","last_change":"2024-09-15T23:58:11.153181+0000","last_active":"2024-09-15T23:58:12.136810+0000","last_peered":"2024-09-15T23:58:12.136810+0000","last_clean":"2024-09-15T23:58:12.136810+0000","last_became_active":"2024-09-15T23:58:11.152920+0000","last_became_peered":"2024-09-15T23:58:11.152920+0000","last_unstale":"2024-09-15T23:58:12.136810+0000","last_undegraded":"2024-09-15T23:58:12.136810+0000","last_fullsized":"2024-09-15T23:58:12.136810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:15.101157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:35.952970+0000","last_change":"2024-09-15T23:58:09.120470+0000","last_active":"2024-09-15T23:58:35.952970+0000","last_peered":"2024-09-15T23:58:35.952970+0000","last_clean":"2024-09-15T23:58:35.952970+0000","last_became_active":"2024-09-15T23:58:09.120143+0000","last_became_peered":"2024-09-15T23:58:09.120143+0000","last_unstale":"2024-09-15T23:58:35.952970+0000","last_undegraded":"2024-09-15T23:58:35.952970+0000","last_fullsized":"2024-09-15T23:58:35.952970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:41.285326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63172,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":170,"num_read_kb":157,"num_write":236,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":189516,"data_compressed":5418,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244980,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1064779,"data_compressed":37506,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972381,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":417323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1010221,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1009506,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":416865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":474530,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":474703,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":418379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58502,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":58329,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":58093,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:58:41.469 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T23:58:41.470 INFO:tasks.cephadm:Setting up client nodes... 2024-09-15T23:58:41.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-15T23:58:41.730 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:42.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:41 smithi031 ceph-mon[31203]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:42.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 -- 172.21.15.31:0/2284897837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 msgr2=0x7f4688108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:42.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 --2- 172.21.15.31:0/2284897837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f4688108f20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ca0 tx=0x7f468002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 -- 172.21.15.31:0/2284897837 shutdown_connections 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 --2- 172.21.15.31:0/2284897837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f4688108f20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 --2- 172.21.15.31:0/2284897837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688101cd0 0x7f4688102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 --2- 172.21.15.31:0/2284897837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4688100ad0 0x7f4688100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 -- 172.21.15.31:0/2284897837 >> 172.21.15.31:0/2284897837 conn(0x7f46880fc280 msgr2=0x7f46880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 -- 172.21.15.31:0/2284897837 shutdown_connections 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.217+0000 7f468f11a640 1 -- 172.21.15.31:0/2284897837 wait complete. 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 Processor -- start 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 -- start start 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688100ad0 0x7f468819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4688101cd0 0x7f468819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:42.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f468810c6b0 con 0x7f4688102690 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f468810c530 con 0x7f4688101cd0 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468f11a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f468810c830 con 0x7f4688100ad0 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468d6b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468d6b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56556/0 (socket says 172.21.15.31:56556) 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468d6b7640 1 -- 172.21.15.31:0/3560872062 learned_addr learned my addr 172.21.15.31:0/3560872062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.218+0000 7f468ceb6640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688100ad0 0x7f468819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 -- 172.21.15.31:0/3560872062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688100ad0 msgr2=0x7f468819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688100ad0 0x7f468819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 -- 172.21.15.31:0/3560872062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4688101cd0 msgr2=0x7f468819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4688101cd0 0x7f468819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 -- 172.21.15.31:0/3560872062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881a1cb0 con 0x7f4688102690 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468d6b7640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4680009dc0 tx=0x7f4680003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:42.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680032ec0 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46800044f0 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46881a1f40 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680030d40 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.219+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881a2450 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.220+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46881a2680 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.221+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4680004040 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.222+0000 7f4685ffb640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 0x7f46700a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.222+0000 7f468ceb6640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 0x7f46700a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.222+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f46800f0f80 con 0x7f4688102690 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.223+0000 7f468ceb6640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 0x7f46700a5960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4688101b30 tx=0x7f4678009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:42.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.225+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46800bde00 con 0x7f4688102690 2024-09-15T23:58:42.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:41 smithi081 ceph-mon[35060]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:42.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:41 smithi102 ceph-mon[34458]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:58:42.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.469+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f46881a28f0 con 0x7f4688102690 2024-09-15T23:58:42.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.471+0000 7f4685ffb640 1 -- 172.21.15.31:0/3560872062 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f46800bd700 con 0x7f4688102690 2024-09-15T23:58:42.473 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-15T23:58:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCydOdmNLEbHBAA0D5eKfKHbM2SBsOEUNy4tA== 2024-09-15T23:58:42.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 msgr2=0x7f46700a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 0x7f46700a5960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4688101b30 tx=0x7f4678009110 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 msgr2=0x7f46881a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4680009dc0 tx=0x7f4680003ba0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 shutdown_connections 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f46700a34a0 0x7f46700a5960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4688102690 0x7f46881a1640 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4688101cd0 0x7f468819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 --2- 172.21.15.31:0/3560872062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4688100ad0 0x7f468819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.473+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 >> 172.21.15.31:0/3560872062 conn(0x7f46880fc280 msgr2=0x7f468806a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.474+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 shutdown_connections 2024-09-15T23:58:42.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:42.474+0000 7f468f11a640 1 -- 172.21.15.31:0/3560872062 wait complete. 2024-09-15T23:58:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:42 smithi031 ceph-mon[31203]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-15T23:58:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:42 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T23:58:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:42 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T23:58:43.285 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-15T23:58:43.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-15T23:58:43.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T23:58:43.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:42 smithi081 ceph-mon[35060]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-15T23:58:43.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:42 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T23:58:43.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:42 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T23:58:43.323 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T23:58:43.324 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-15T23:58:43.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-15T23:58:43.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:42 smithi102 ceph-mon[34458]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-15T23:58:43.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:42 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T23:58:43.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:42 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3560872062' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T23:58:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:44.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 -- 172.21.15.31:0/1386509835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8103660 msgr2=0x7f58c810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:44.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 --2- 172.21.15.31:0/1386509835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8103660 0x7f58c810bb50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009900 tx=0x7f58b802a480 comp rx=0 tx=0).stop 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 -- 172.21.15.31:0/1386509835 shutdown_connections 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 --2- 172.21.15.31:0/1386509835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 0x7f58c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 --2- 172.21.15.31:0/1386509835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8103660 0x7f58c810bb50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 --2- 172.21.15.31:0/1386509835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8102d40 0x7f58c8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 -- 172.21.15.31:0/1386509835 >> 172.21.15.31:0/1386509835 conn(0x7f58c80fc7a0 msgr2=0x7f58c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 -- 172.21.15.31:0/1386509835 shutdown_connections 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.137+0000 7f58d06c1640 1 -- 172.21.15.31:0/1386509835 wait complete. 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 Processor -- start 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 -- start start 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8103660 0x7f58c8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:44.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 0x7f58c8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8111ee0 con 0x7f58c8102d40 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8111d60 con 0x7f58c8103660 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58d06c1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8112060 con 0x7f58c810c170 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.138+0000 7f58ce45d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56572/0 (socket says 172.21.15.31:56572) 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 -- 172.21.15.31:0/2085413328 learned_addr learned my addr 172.21.15.31:0/2085413328 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58cdc5c640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8103660 0x7f58c8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58cec5e640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 0x7f58c8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 -- 172.21.15.31:0/2085413328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 msgr2=0x7f58c8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 0x7f58c8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 -- 172.21.15.31:0/2085413328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8103660 msgr2=0x7f58c8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:44.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8103660 0x7f58c8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 -- 172.21.15.31:0/2085413328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c81a69d0 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58ce45d640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58c400b4a0 tx=0x7f58c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4014070 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c81a6cc0 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58c4010ac0 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4017c30 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.139+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c81a71c0 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.140+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c81031a0 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.142+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f58c4017070 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.142+0000 7f58bf7fe640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 0x7f58b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.142+0000 7f58cdc5c640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 0x7f58b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.142+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f58c40cee00 con 0x7f58c8102d40 2024-09-15T23:58:44.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.143+0000 7f58cdc5c640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 0x7f58b40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58b8031440 tx=0x7f58b8005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:44.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.146+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58c409b630 con 0x7f58c8102d40 2024-09-15T23:58:44.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.369+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f58c802d7c0 con 0x7f58c8102d40 2024-09-15T23:58:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.374+0000 7f58bf7fe640 1 -- 172.21.15.31:0/2085413328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264046 (secure 0 0 0) 0x7f58c409f930 con 0x7f58c8102d40 2024-09-15T23:58:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.377+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 msgr2=0x7f58b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.377+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 0x7f58b40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58b8031440 tx=0x7f58b8005e80 comp rx=0 tx=0).stop 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.377+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 msgr2=0x7f58c81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.377+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58c400b4a0 tx=0x7f58c400b960 comp rx=0 tx=0).stop 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 shutdown_connections 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f58b40a32d0 0x7f58b40a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58c810c170 0x7f58c8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58c8103660 0x7f58c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 --2- 172.21.15.31:0/2085413328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58c8102d40 0x7f58c81009a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 >> 172.21.15.31:0/2085413328 conn(0x7f58c80fc7a0 msgr2=0x7f58c80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 shutdown_connections 2024-09-15T23:58:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:44.378+0000 7f58d06c1640 1 -- 172.21.15.31:0/2085413328 wait complete. 2024-09-15T23:58:45.151 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":797003742},{"type":"v1","addr":"172.21.15.31:6801","nonce":797003742}]},"active_addr":"172.21.15.31:6801/797003742","active_change":"2024-09-15T23:51:11.105564+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2677380564}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3296404867}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":540718575}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1147118595}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2617564247}]}]} 2024-09-15T23:58:45.156 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-15T23:58:45.157 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-15T23:58:45.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:58:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:44 smithi031 ceph-mon[31203]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T23:58:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:44 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2085413328' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T23:58:45.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:44 smithi081 ceph-mon[35060]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T23:58:45.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:44 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2085413328' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T23:58:45.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:44 smithi102 ceph-mon[34458]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T23:58:45.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:44 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2085413328' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T23:58:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 -- 172.21.15.31:0/3670980216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102ef0 msgr2=0x7f345c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 --2- 172.21.15.31:0/3670980216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102ef0 0x7f345c10da40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3454009ca0 tx=0x7f345402a950 comp rx=0 tx=0).stop 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 -- 172.21.15.31:0/3670980216 shutdown_connections 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 --2- 172.21.15.31:0/3670980216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102ef0 0x7f345c10da40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 --2- 172.21.15.31:0/3670980216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c102570 0x7f345c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 --2- 172.21.15.31:0/3670980216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c100e20 0x7f345c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 -- 172.21.15.31:0/3670980216 >> 172.21.15.31:0/3670980216 conn(0x7f345c0fc7d0 msgr2=0x7f345c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 -- 172.21.15.31:0/3670980216 shutdown_connections 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 -- 172.21.15.31:0/3670980216 wait complete. 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.940+0000 7f346151d640 1 Processor -- start 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 -- start start 2024-09-15T23:58:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c100e20 0x7f345c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c102ef0 0x7f345c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f345c110ed0 con 0x7f345c102570 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f345c110d50 con 0x7f345c100e20 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f346151d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f345c111050 con 0x7f345c102ef0 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56600/0 (socket says 172.21.15.31:56600) 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 -- 172.21.15.31:0/3372868158 learned_addr learned my addr 172.21.15.31:0/3372868158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f345affd640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c100e20 0x7f345c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f345b7fe640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c102ef0 0x7f345c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 -- 172.21.15.31:0/3372868158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c102ef0 msgr2=0x7f345c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c102ef0 0x7f345c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 -- 172.21.15.31:0/3372868158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c100e20 msgr2=0x7f345c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c100e20 0x7f345c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 -- 172.21.15.31:0/3372868158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c19f6f0 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.941+0000 7f3453fff640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f344400b4a0 tx=0x7f344400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.942+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444015020 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.942+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3444011ce0 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.942+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c19f9e0 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.942+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444010dc0 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.942+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c1a72c0 con 0x7f345c102570 2024-09-15T23:58:45.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.943+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c101280 con 0x7f345c102570 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.944+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3444018020 con 0x7f345c102570 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.945+0000 7f3458ff9640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 0x7f34480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.945+0000 7f345affd640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 0x7f34480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.945+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f34440cf660 con 0x7f345c102570 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.945+0000 7f345affd640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 0x7f34480a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:45.948+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f344401f020 con 0x7f345c102570 2024-09-15T23:58:46.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.126+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f345c1a7550 con 0x7f345c102570 2024-09-15T23:58:46.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.127+0000 7f3458ff9640 1 -- 172.21.15.31:0/3372868158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7f344409c4e0 con 0x7f345c102570 2024-09-15T23:58:46.129 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:46.129 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:58:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.129+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 msgr2=0x7f34480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.129+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 0x7f34480a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009110 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.129+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 msgr2=0x7f345c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.129+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f344400b4a0 tx=0x7f344400b8a0 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 shutdown_connections 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f34480a32d0 0x7f34480a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f345c102ef0 0x7f345c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f345c102570 0x7f345c19ac80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 --2- 172.21.15.31:0/3372868158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f345c100e20 0x7f345c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 >> 172.21.15.31:0/3372868158 conn(0x7f345c0fc7d0 msgr2=0x7f345c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 shutdown_connections 2024-09-15T23:58:46.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:46.130+0000 7f346151d640 1 -- 172.21.15.31:0/3372868158 wait complete. 2024-09-15T23:58:46.762 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-15T23:58:46.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:58:47.032 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:47.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:46 smithi031 ceph-mon[31203]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:47.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:46 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3372868158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:47.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:46 smithi081 ceph-mon[35060]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:47.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:46 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3372868158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:47.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:46 smithi102 ceph-mon[34458]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:47.372 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3372868158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 -- 172.21.15.31:0/3634488077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400103740 msgr2=0x7f1400107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 --2- 172.21.15.31:0/3634488077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400103740 0x7f1400107b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009930 tx=0x7f13f002a420 comp rx=0 tx=0).stop 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 -- 172.21.15.31:0/3634488077 shutdown_connections 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 --2- 172.21.15.31:0/3634488077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400108290 0x7f140010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 --2- 172.21.15.31:0/3634488077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400103740 0x7f1400107b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 --2- 172.21.15.31:0/3634488077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 -- 172.21.15.31:0/3634488077 >> 172.21.15.31:0/3634488077 conn(0x7f14000fc7d0 msgr2=0x7f14000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 -- 172.21.15.31:0/3634488077 shutdown_connections 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.556+0000 7f1405fb9640 1 -- 172.21.15.31:0/3634488077 wait complete. 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 Processor -- start 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 -- start start 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400103740 0x7f1400198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1400111fe0 con 0x7f1400108290 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1400111e60 con 0x7f1400102d90 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.557+0000 7f1405fb9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1400112160 con 0x7f1400103740 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56618/0 (socket says 172.21.15.31:56618) 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 -- 172.21.15.31:0/2509926860 learned_addr learned my addr 172.21.15.31:0/2509926860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13ff7fe640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13feffd640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400103740 0x7f1400198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 -- 172.21.15.31:0/2509926860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400103740 msgr2=0x7f1400198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400103740 0x7f1400198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 -- 172.21.15.31:0/2509926860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 msgr2=0x7f1400198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 -- 172.21.15.31:0/2509926860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140019d5f0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13ff7fe640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fffff640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13f800b970 tx=0x7f13f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8019070 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f8014440 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140019d8e0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80075d0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.558+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14001a51c0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.560+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140006a7f0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.560+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f13f801c070 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.561+0000 7f13fcff9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 0x7f13ec0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.561+0000 7f13ff7fe640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 0x7f13ec0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.562+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f13f80cedd0 con 0x7f1400108290 2024-09-15T23:58:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.562+0000 7f13ff7fe640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 0x7f13ec0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009930 tx=0x7f13e8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.565+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13f809bc50 con 0x7f1400108290 2024-09-15T23:58:47.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.741+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f140010a2a0 con 0x7f1400108290 2024-09-15T23:58:47.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.742+0000 7f13fcff9640 1 -- 172.21.15.31:0/2509926860 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7f13f809b550 con 0x7f1400108290 2024-09-15T23:58:47.744 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:58:47.744 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:58:47.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 msgr2=0x7f13ec0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:47.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 0x7f13ec0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009930 tx=0x7f13e8008040 comp rx=0 tx=0).stop 2024-09-15T23:58:47.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 msgr2=0x7f140019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:47.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13f800b970 tx=0x7f13f800be30 comp rx=0 tx=0).stop 2024-09-15T23:58:47.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 shutdown_connections 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f13ec0a3350 0x7f13ec0a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1400108290 0x7f140019cf10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400103740 0x7f1400198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 --2- 172.21.15.31:0/2509926860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1400102d90 0x7f1400198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.744+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 >> 172.21.15.31:0/2509926860 conn(0x7f14000fc7d0 msgr2=0x7f1400105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.745+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 shutdown_connections 2024-09-15T23:58:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:47.745+0000 7f1405fb9640 1 -- 172.21.15.31:0/2509926860 wait complete. 2024-09-15T23:58:48.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:48 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2509926860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:48.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:48 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2509926860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:48.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:48 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2509926860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:58:48.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-15T23:58:48.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-15T23:58:48.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-15T23:58:48.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-15T23:58:48.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-15T23:58:48.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-15T23:58:48.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-15T23:58:48.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-15T23:58:48.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-15T23:58:48.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-15T23:58:48.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-15T23:58:48.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-15T23:58:49.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:49 smithi031 ceph-mon[31203]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:49.249 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:49.264 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:49.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:49 smithi081 ceph-mon[35060]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:49.364 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:49 smithi102 ceph-mon[34458]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 -- 172.21.15.31:0/986202315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08102da0 msgr2=0x7fad08103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 --2- 172.21.15.31:0/986202315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08102da0 0x7fad08103180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fad04009900 tx=0x7fad0402a480 comp rx=0 tx=0).stop 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 -- 172.21.15.31:0/986202315 shutdown_connections 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 --2- 172.21.15.31:0/986202315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad0810c180 0x7fad0810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 --2- 172.21.15.31:0/986202315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad081036c0 0x7fad0810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 --2- 172.21.15.31:0/986202315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08102da0 0x7fad08103180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 -- 172.21.15.31:0/986202315 >> 172.21.15.31:0/986202315 conn(0x7fad080fc7a0 msgr2=0x7fad080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 -- 172.21.15.31:0/986202315 shutdown_connections 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.030+0000 7fad10572640 1 -- 172.21.15.31:0/986202315 wait complete. 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 Processor -- start 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 -- start start 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad08102da0 0x7fad08198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 0x7fad081987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad0810c180 0x7fad0819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad08111e10 con 0x7fad081036c0 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad08111c90 con 0x7fad0810c180 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad10572640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad08111f90 con 0x7fad08102da0 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 0x7fad081987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 0x7fad081987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56638/0 (socket says 172.21.15.31:56638) 2024-09-15T23:58:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 -- 172.21.15.31:0/2522465970 learned_addr learned my addr 172.21.15.31:0/2522465970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0e30e640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad08102da0 0x7fad08198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0eb0f640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad0810c180 0x7fad0819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 -- 172.21.15.31:0/2522465970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad08102da0 msgr2=0x7fad08198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad08102da0 0x7fad08198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 -- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad0810c180 msgr2=0x7fad0819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad0810c180 0x7fad0819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.031+0000 7fad0db0d640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0819d240 con 0x7fad081036c0 2024-09-15T23:58:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7fad0db0d640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 0x7fad081987d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facf800c920 tx=0x7facf800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8007c70 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf8007e10 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8018820 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0819d530 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.032+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a4e10 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.033+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad08062980 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.034+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7facf80189c0 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7facff7fe640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7facf40a3350 0x7facf40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7fad0e30e640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7facf40a3350 0x7facf40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7facf80cfe30 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7facff7fe640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] conn(0x7facf40acba0 0x7facf40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facf8015e70 con 0x7facf40acba0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7facf80d0310 con 0x7fad081036c0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7fad0e30e640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7facf40a3350 0x7facf40a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad04031440 tx=0x7fad04005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.035+0000 7fad0eb0f640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] conn(0x7facf40acba0 0x7facf40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.036+0000 7fad0eb0f640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] conn(0x7facf40acba0 0x7facf40aefc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.036+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== osd.10 v2:172.21.15.81:6824/397827149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf8015e70 con 0x7facf40acba0 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.050+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 --> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad08062d80 con 0x7facf40acba0 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7facff7fe640 1 -- 172.21.15.31:0/2522465970 <== osd.10 v2:172.21.15.81:6824/397827149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad08062d80 con 0x7facf40acba0 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] conn(0x7facf40acba0 msgr2=0x7facf40aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] conn(0x7facf40acba0 0x7facf40aefc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7facf40a3350 msgr2=0x7facf40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7facf40a3350 0x7facf40a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad04031440 tx=0x7fad04005e80 comp rx=0 tx=0).stop 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 msgr2=0x7fad081987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 --2- 172.21.15.31:0/2522465970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad081036c0 0x7fad081987d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facf800c920 tx=0x7facf800cde0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad0eb0f640 1 -- 172.21.15.31:0/2522465970 reap_dead start 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 shutdown_connections 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 >> 172.21.15.31:0/2522465970 conn(0x7fad080fc7a0 msgr2=0x7fad080fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:50.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 shutdown_connections 2024-09-15T23:58:50.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.051+0000 7fad10572640 1 -- 172.21.15.31:0/2522465970 wait complete. 2024-09-15T23:58:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1823569562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 msgr2=0x7ff7e81031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1823569562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e81031e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009900 tx=0x7ff7d802a480 comp rx=0 tx=0).stop 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1823569562 shutdown_connections 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1823569562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7e810c1a0 0x7ff7e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1823569562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e8103720 0x7ff7e810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1823569562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e81031e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1823569562 >> 172.21.15.31:0/1823569562 conn(0x7ff7e80fc7a0 msgr2=0x7ff7e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1823569562 shutdown_connections 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.466+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1823569562 wait complete. 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 Processor -- start 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 -- start start 2024-09-15T23:58:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7e8103720 0x7ff7e81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e810c1a0 0x7ff7e819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8111e60 con 0x7ff7e8102e00 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8111ce0 con 0x7ff7e810c1a0 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7ecc21640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8111fe0 con 0x7ff7e8103720 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7e659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7e659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56654/0 (socket says 172.21.15.31:56654) 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.467+0000 7ff7e659c640 1 -- 172.21.15.31:0/1994700883 learned_addr learned my addr 172.21.15.31:0/1994700883 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7ddd9b640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7e8103720 0x7ff7e81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e6d9d640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e810c1a0 0x7ff7e819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 -- 172.21.15.31:0/1994700883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7e8103720 msgr2=0x7ff7e81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7e8103720 0x7ff7e81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 -- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e810c1a0 msgr2=0x7ff7e819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e810c1a0 0x7ff7e819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e819d310 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7e659c640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e8198360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8006010 tx=0x7ff7d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d803f030 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e819d5a0 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7d80035f0 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8035070 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.468+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e8100990 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.469+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7e8062980 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.470+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff7d8003b10 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7dffff640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff7d40a76e0 0x7ff7d40a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff7d80f0ce0 con 0x7ff7e8102e00 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7ddd9b640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff7d40a76e0 0x7ff7d40a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7dffff640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] conn(0x7ff7d40b0f30 0x7ff7d40b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7d8001d30 con 0x7ff7d40b0f30 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.471+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff7d80f1090 con 0x7ff7e8102e00 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.472+0000 7ff7ddd9b640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff7d40a76e0 0x7ff7d40a9ba0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80feda0 tx=0x7ff7d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.472+0000 7ff7e6d9d640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] conn(0x7ff7d40b0f30 0x7ff7d40b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.472+0000 7ff7e6d9d640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] conn(0x7ff7d40b0f30 0x7ff7d40b3350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.473+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== osd.7 v2:172.21.15.81:6816/326400233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7d8001d30 con 0x7ff7d40b0f30 2024-09-15T23:58:50.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.487+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 --> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7e8062d80 con 0x7ff7d40b0f30 2024-09-15T23:58:50.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7dffff640 1 -- 172.21.15.31:0/1994700883 <== osd.7 v2:172.21.15.81:6816/326400233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff7e8062d80 con 0x7ff7d40b0f30 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] conn(0x7ff7d40b0f30 msgr2=0x7ff7d40b3350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] conn(0x7ff7d40b0f30 0x7ff7d40b3350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff7d40a76e0 msgr2=0x7ff7d40a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff7d40a76e0 0x7ff7d40a9ba0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80feda0 tx=0x7ff7d0008040 comp rx=0 tx=0).stop 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 msgr2=0x7ff7e8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 --2- 172.21.15.31:0/1994700883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e8102e00 0x7ff7e8198360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8006010 tx=0x7ff7d8004270 comp rx=0 tx=0).stop 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7e6d9d640 1 -- 172.21.15.31:0/1994700883 reap_dead start 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 shutdown_connections 2024-09-15T23:58:50.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 >> 172.21.15.31:0/1994700883 conn(0x7ff7e80fc7a0 msgr2=0x7ff7e80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:50.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 shutdown_connections 2024-09-15T23:58:50.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:50.488+0000 7ff7ecc21640 1 -- 172.21.15.31:0/1994700883 wait complete. 2024-09-15T23:58:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:51 smithi031 ceph-mon[31203]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-15T23:58:51.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:51 smithi081 ceph-mon[35060]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-15T23:58:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:51 smithi102 ceph-mon[34458]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-15T23:58:51.385 INFO:teuthology.orchestra.run.smithi031.stdout:206158430249 2024-09-15T23:58:51.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 -- 172.21.15.31:0/3645321396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102ef0 msgr2=0x7f48bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 --2- 172.21.15.31:0/3645321396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102ef0 0x7f48bc10da40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b402a480 comp rx=0 tx=0).stop 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 -- 172.21.15.31:0/3645321396 shutdown_connections 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 --2- 172.21.15.31:0/3645321396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102ef0 0x7f48bc10da40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 --2- 172.21.15.31:0/3645321396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48bc102570 0x7f48bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 --2- 172.21.15.31:0/3645321396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc100e20 0x7f48bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 -- 172.21.15.31:0/3645321396 >> 172.21.15.31:0/3645321396 conn(0x7f48bc0fc7d0 msgr2=0x7f48bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:52.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 -- 172.21.15.31:0/3645321396 shutdown_connections 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.634+0000 7f48c35c5640 1 -- 172.21.15.31:0/3645321396 wait complete. 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 Processor -- start 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 -- start start 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48bc100e20 0x7f48bc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 0x7f48bc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc102ef0 0x7f48bc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc110e30 con 0x7f48bc102570 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc110cb0 con 0x7f48bc102ef0 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c35c5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc110fb0 con 0x7f48bc100e20 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c0b60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 0x7f48bc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c0b60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 0x7f48bc198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58734/0 (socket says 172.21.15.31:58734) 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c0b60640 1 -- 172.21.15.31:0/897256610 learned_addr learned my addr 172.21.15.31:0/897256610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:52.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c1361640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48bc100e20 0x7f48bc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.635+0000 7f48c1b62640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc102ef0 0x7f48bc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 -- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48bc100e20 msgr2=0x7f48bc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48bc100e20 0x7f48bc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 -- 172.21.15.31:0/897256610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc102ef0 msgr2=0x7f48bc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc102ef0 0x7f48bc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc19d390 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c1b62640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48bc102ef0 0x7f48bc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c0b60640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 0x7f48bc198920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e9f0 tx=0x7f48b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b000cd80 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48b00054d0 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc19d680 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0010490 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.636+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc1a4f60 con 0x7f48bc102570 2024-09-15T23:58:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.637+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48bc104300 con 0x7f48bc102570 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.638+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f48b0005000 con 0x7f48bc102570 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48ba7fc640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48a40a3350 0x7f48a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48c1361640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48a40a3350 0x7f48a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f48b00d0070 con 0x7f48bc102570 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48ba7fc640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] conn(0x7f48a40acba0 0x7f48a40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48b001aec0 con 0x7f48a40acba0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f48b00d0550 con 0x7f48bc102570 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48c1361640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48a40a3350 0x7f48a40a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f48ac005fd0 tx=0x7f48ac009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.639+0000 7f48c1b62640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] conn(0x7f48a40acba0 0x7f48a40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.640+0000 7f48c1b62640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] conn(0x7f48a40acba0 0x7f48a40aefc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.640+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== osd.8 v2:172.21.15.102:6816/3203076122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48b001aec0 con 0x7f48a40acba0 2024-09-15T23:58:52.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.654+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 --> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48bc1021b0 con 0x7f48a40acba0 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48ba7fc640 1 -- 172.21.15.31:0/897256610 <== osd.8 v2:172.21.15.102:6816/3203076122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48bc1021b0 con 0x7f48a40acba0 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] conn(0x7f48a40acba0 msgr2=0x7f48a40aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] conn(0x7f48a40acba0 0x7f48a40aefc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48a40a3350 msgr2=0x7f48a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48a40a3350 0x7f48a40a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f48ac005fd0 tx=0x7f48ac009760 comp rx=0 tx=0).stop 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 msgr2=0x7f48bc198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 --2- 172.21.15.31:0/897256610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48bc102570 0x7f48bc198920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e9f0 tx=0x7f48b000edf0 comp rx=0 tx=0).stop 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c1b62640 1 -- 172.21.15.31:0/897256610 reap_dead start 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 shutdown_connections 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.655+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 >> 172.21.15.31:0/897256610 conn(0x7f48bc0fc7d0 msgr2=0x7f48bc1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.656+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 shutdown_connections 2024-09-15T23:58:52.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.656+0000 7f48c35c5640 1 -- 172.21.15.31:0/897256610 wait complete. 2024-09-15T23:58:53.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 -- 172.21.15.31:0/3890669768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08102790 msgr2=0x7f3b081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 --2- 172.21.15.31:0/3890669768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08102790 0x7f3b081067e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009900 tx=0x7f3b0402a410 comp rx=0 tx=0).stop 2024-09-15T23:58:53.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 -- 172.21.15.31:0/3890669768 shutdown_connections 2024-09-15T23:58:53.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 --2- 172.21.15.31:0/3890669768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 0x7f3b0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 --2- 172.21.15.31:0/3890669768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08102790 0x7f3b081067e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 --2- 172.21.15.31:0/3890669768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b08101de0 0x7f3b081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 -- 172.21.15.31:0/3890669768 >> 172.21.15.31:0/3890669768 conn(0x7f3b080fd8c0 msgr2=0x7f3b080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 -- 172.21.15.31:0/3890669768 shutdown_connections 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.830+0000 7f3b0f307640 1 -- 172.21.15.31:0/3890669768 wait complete. 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 Processor -- start 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 -- start start 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 0x7f3b0819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b08102790 0x7f3b0819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 0x7f3b0819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111e40 con 0x7f3b08101de0 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111cc0 con 0x7f3b08102790 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0f307640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08111fc0 con 0x7f3b08106f10 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.831+0000 7f3b0d0a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 0x7f3b0819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 0x7f3b0819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58752/0 (socket says 172.21.15.31:58752) 2024-09-15T23:58:53.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 -- 172.21.15.31:0/2823678836 learned_addr learned my addr 172.21.15.31:0/2823678836 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:53.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0c8a2640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b08102790 0x7f3b0819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d8a4640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 0x7f3b0819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 -- 172.21.15.31:0/2823678836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 msgr2=0x7f3b0819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 0x7f3b0819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 -- 172.21.15.31:0/2823678836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b08102790 msgr2=0x7f3b0819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b08102790 0x7f3b0819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0819f5b0 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d8a4640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b08106f10 0x7f3b0819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0d0a3640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 0x7f3b0819a600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c920 tx=0x7f3afc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007c70 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0819f8a0 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3afc007e10 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc018820 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.832+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b081a7180 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.833+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b08075700 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.834+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3afc0189c0 con 0x7f3b08101de0 2024-09-15T23:58:53.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3afa7fc640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3af40a32d0 0x7f3af40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3b0c8a2640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3af40a32d0 0x7f3af40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3afc0cfee0 con 0x7f3b08101de0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3afa7fc640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] conn(0x7f3af40acb20 0x7f3af40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3afc015e70 con 0x7f3af40acb20 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3afc0d03c0 con 0x7f3b08101de0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.835+0000 7f3b0d8a4640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] conn(0x7f3af40acb20 0x7f3af40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.836+0000 7f3b0c8a2640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3af40a32d0 0x7f3af40a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3b04031440 tx=0x7f3b04005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.836+0000 7f3b0d8a4640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] conn(0x7f3af40acb20 0x7f3af40aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.836+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== osd.3 v2:172.21.15.31:6810/487246469 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3afc015e70 con 0x7f3af40acb20 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.850+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 --> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b081032c0 con 0x7f3af40acb20 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.850+0000 7f3afa7fc640 1 -- 172.21.15.31:0/2823678836 <== osd.3 v2:172.21.15.31:6810/487246469 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b081032c0 con 0x7f3af40acb20 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] conn(0x7f3af40acb20 msgr2=0x7f3af40aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] conn(0x7f3af40acb20 0x7f3af40aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3af40a32d0 msgr2=0x7f3af40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3af40a32d0 0x7f3af40a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3b04031440 tx=0x7f3b04005e80 comp rx=0 tx=0).stop 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 msgr2=0x7f3b0819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 --2- 172.21.15.31:0/2823678836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b08101de0 0x7f3b0819a600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c920 tx=0x7f3afc00cde0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0d8a4640 1 -- 172.21.15.31:0/2823678836 reap_dead start 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 shutdown_connections 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 >> 172.21.15.31:0/2823678836 conn(0x7f3b080fd8c0 msgr2=0x7f3b080fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 shutdown_connections 2024-09-15T23:58:53.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.851+0000 7f3b0f307640 1 -- 172.21.15.31:0/2823678836 wait complete. 2024-09-15T23:58:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:53 smithi031 ceph-mon[31203]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-15T23:58:53.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:53 smithi081 ceph-mon[35060]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-15T23:58:53.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:53 smithi102 ceph-mon[34458]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-15T23:58:53.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.674+0000 7f20ab6ee640 1 -- 172.21.15.31:0/2420514543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4101de0 msgr2=0x7f20a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.674+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/2420514543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4101de0 0x7f20a41021c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2094009ca0 tx=0x7f209402a8c0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 -- 172.21.15.31:0/2420514543 shutdown_connections 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/2420514543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4106f10 0x7f20a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/2420514543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a4102790 0x7f20a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/2420514543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4101de0 0x7f20a41021c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 -- 172.21.15.31:0/2420514543 >> 172.21.15.31:0/2420514543 conn(0x7f20a40fd8c0 msgr2=0x7f20a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 -- 172.21.15.31:0/2420514543 shutdown_connections 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 -- 172.21.15.31:0/2420514543 wait complete. 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 Processor -- start 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.675+0000 7f20ab6ee640 1 -- start start 2024-09-15T23:58:53.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a4101de0 0x7f20a4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4102790 0x7f20a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 0x7f20a419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111e90 con 0x7f20a4106f10 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4111d10 con 0x7f20a4101de0 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4112010 con 0x7f20a4102790 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 0x7f20a419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 0x7f20a419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58774/0 (socket says 172.21.15.31:58774) 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 -- 172.21.15.31:0/3500544309 learned_addr learned my addr 172.21.15.31:0/3500544309 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a948a640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a4101de0 0x7f20a4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a8c89640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4102790 0x7f20a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 -- 172.21.15.31:0/3500544309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4102790 msgr2=0x7f20a4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4102790 0x7f20a4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 -- 172.21.15.31:0/3500544309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a4101de0 msgr2=0x7f20a4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a4101de0 0x7f20a4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a419d3c0 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a9c8b640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 0x7f20a419cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fd0 tx=0x7f209c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c011830 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c016070 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c010de0 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a419d6b0 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.676+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a4f90 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.677+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20a4062900 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f209c0119d0 con 0x7f20a4106f10 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f208c0a3350 0x7f208c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a948a640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f208c0a3350 0x7f208c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f209c0cf7d0 con 0x7f20a4106f10 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] conn(0x7f208c0acba0 0x7f208c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f209c004450 con 0x7f208c0acba0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f209c0cfcb0 con 0x7f20a4106f10 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a8c89640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] conn(0x7f208c0acba0 0x7f208c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a948a640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f208c0a3350 0x7f208c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f2094031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.680+0000 7f20a8c89640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] conn(0x7f208c0acba0 0x7f208c0aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.681+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== osd.0 v2:172.21.15.31:6802/1628537421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f209c004450 con 0x7f208c0acba0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.694+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 --> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20a4062d00 con 0x7f208c0acba0 2024-09-15T23:58:53.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.694+0000 7f20a27fc640 1 -- 172.21.15.31:0/3500544309 <== osd.0 v2:172.21.15.31:6802/1628537421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20a4062d00 con 0x7f208c0acba0 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] conn(0x7f208c0acba0 msgr2=0x7f208c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] conn(0x7f208c0acba0 0x7f208c0aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f208c0a3350 msgr2=0x7f208c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f208c0a3350 0x7f208c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f2094031000 comp rx=0 tx=0).stop 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 msgr2=0x7f20a419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 --2- 172.21.15.31:0/3500544309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20a4106f10 0x7f20a419cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fd0 tx=0x7f209c00b8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20a9c8b640 1 -- 172.21.15.31:0/3500544309 reap_dead start 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 shutdown_connections 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 >> 172.21.15.31:0/3500544309 conn(0x7f20a40fd8c0 msgr2=0x7f20a40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 shutdown_connections 2024-09-15T23:58:53.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.695+0000 7f20ab6ee640 1 -- 172.21.15.31:0/3500544309 wait complete. 2024-09-15T23:58:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 -- 172.21.15.31:0/1360112845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd808103720 msgr2=0x7fd80810bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 --2- 172.21.15.31:0/1360112845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd808103720 0x7fd80810bc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009900 tx=0x7fd7fc02a480 comp rx=0 tx=0).stop 2024-09-15T23:58:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 -- 172.21.15.31:0/1360112845 shutdown_connections 2024-09-15T23:58:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 --2- 172.21.15.31:0/1360112845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80810c1a0 0x7fd80810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 --2- 172.21.15.31:0/1360112845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd808103720 0x7fd80810bc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 --2- 172.21.15.31:0/1360112845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd808102e00 0x7fd8081031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 -- 172.21.15.31:0/1360112845 >> 172.21.15.31:0/1360112845 conn(0x7fd8080fc7a0 msgr2=0x7fd8080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 -- 172.21.15.31:0/1360112845 shutdown_connections 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.997+0000 7fd80fc66640 1 -- 172.21.15.31:0/1360112845 wait complete. 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 Processor -- start 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 -- start start 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd808102e00 0x7fd808198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd808103720 0x7fd8081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 0x7fd80819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd808111e70 con 0x7fd80810c1a0 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd808111cf0 con 0x7fd808102e00 2024-09-15T23:58:54.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80fc66640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd808111ff0 con 0x7fd808103720 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80e203640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 0x7fd80819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80e203640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 0x7fd80819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58796/0 (socket says 172.21.15.31:58796) 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80e203640 1 -- 172.21.15.31:0/2353102236 learned_addr learned my addr 172.21.15.31:0/2353102236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.998+0000 7fd80da02640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd808102e00 0x7fd808198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80d201640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd808103720 0x7fd8081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 -- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd808103720 msgr2=0x7fd8081988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd808103720 0x7fd8081988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 -- 172.21.15.31:0/2353102236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd808102e00 msgr2=0x7fd808198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd808102e00 0x7fd808198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80819d320 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80e203640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 0x7fd80819cc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800eaa0 tx=0x7fd7f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f800cdb0 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7f8005020 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80116d0 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80819d610 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:53.999+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a4ef0 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.000+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd808062980 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.001+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd7f80051d0 con 0x7fd80810c1a0 2024-09-15T23:58:54.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd806ffd640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7f00a3350 0x7fd7f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd80da02640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7f00a3350 0x7fd7f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd7f80cfae0 con 0x7fd80810c1a0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd806ffd640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] conn(0x7fd7f00acba0 0x7fd7f00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7f8018ec0 con 0x7fd7f00acba0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fd7f80d42a0 con 0x7fd80810c1a0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd80da02640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7f00a3350 0x7fd7f00a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd800005fd0 tx=0x7fd80000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.002+0000 7fd80d201640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] conn(0x7fd7f00acba0 0x7fd7f00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.003+0000 7fd80d201640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] conn(0x7fd7f00acba0 0x7fd7f00aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.005+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== osd.2 v2:172.21.15.102:6800/3162988112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7f8018ec0 con 0x7fd7f00acba0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 --> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd808062e30 con 0x7fd7f00acba0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd806ffd640 1 -- 172.21.15.31:0/2353102236 <== osd.2 v2:172.21.15.102:6800/3162988112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd808062e30 con 0x7fd7f00acba0 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] conn(0x7fd7f00acba0 msgr2=0x7fd7f00aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] conn(0x7fd7f00acba0 0x7fd7f00aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7f00a3350 msgr2=0x7fd7f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd7f00a3350 0x7fd7f00a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd800005fd0 tx=0x7fd80000b390 comp rx=0 tx=0).stop 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 msgr2=0x7fd80819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80fc66640 1 --2- 172.21.15.31:0/2353102236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd80810c1a0 0x7fd80819cc40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800eaa0 tx=0x7fd7f800ef60 comp rx=0 tx=0).stop 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.018+0000 7fd80e203640 1 -- 172.21.15.31:0/2353102236 reap_dead start 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.019+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 shutdown_connections 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.019+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 >> 172.21.15.31:0/2353102236 conn(0x7fd8080fc7a0 msgr2=0x7fd8080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.019+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 shutdown_connections 2024-09-15T23:58:54.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:54.019+0000 7fd80fc66640 1 -- 172.21.15.31:0/2353102236 wait complete. 2024-09-15T23:58:54.825 INFO:teuthology.orchestra.run.smithi031.stdout:77309411396 2024-09-15T23:58:54.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-15T23:58:54.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 -- 172.21.15.31:0/1937353243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 msgr2=0x7f48d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 --2- 172.21.15.31:0/1937353243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d810d950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009cd0 tx=0x7f48d002a8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 -- 172.21.15.31:0/1937353243 shutdown_connections 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 --2- 172.21.15.31:0/1937353243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d810d950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 --2- 172.21.15.31:0/1937353243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8100a80 0x7f48d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 --2- 172.21.15.31:0/1937353243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48d8106a80 0x7f48d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 -- 172.21.15.31:0/1937353243 >> 172.21.15.31:0/1937353243 conn(0x7f48d80fc7a0 msgr2=0x7f48d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 -- 172.21.15.31:0/1937353243 shutdown_connections 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.960+0000 7f48de0a8640 1 -- 172.21.15.31:0/1937353243 wait complete. 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 Processor -- start 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 -- start start 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48d8100a80 0x7f48d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8106a80 0x7f48d81a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8110d30 con 0x7f48d8101420 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8110bb0 con 0x7f48d8106a80 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48de0a8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8110eb0 con 0x7f48d8100a80 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48d6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d8199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58758/0 (socket says 172.21.15.31:58758) 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48d6ffd640 1 -- 172.21.15.31:0/2251696035 learned_addr learned my addr 172.21.15.31:0/2251696035 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.961+0000 7f48d77fe640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48d8100a80 0x7f48d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d7fff640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8106a80 0x7f48d81a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 -- 172.21.15.31:0/2251696035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48d8100a80 msgr2=0x7f48d81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48d8100a80 0x7f48d81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 -- 172.21.15.31:0/2251696035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8106a80 msgr2=0x7f48d81a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8106a80 0x7f48d81a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d81a4780 con 0x7f48d8101420 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d7fff640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8106a80 0x7f48d81a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d6ffd640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d8199700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d950 tx=0x7f48cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc017070 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48cc009c30 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d81a4950 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00f620 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.962+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d81a4e90 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.963+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48d8062900 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.964+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f48cc0109a0 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d4ff9640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48c00a3350 0x7f48c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d77fe640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48c00a3350 0x7f48c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f48cc0cf540 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d4ff9640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] conn(0x7f48c00acba0 0x7f48c00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48cc004470 con 0x7f48c00acba0 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f48cc0d6050 con 0x7f48d8101420 2024-09-15T23:58:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d7fff640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] conn(0x7f48c00acba0 0x7f48c00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.965+0000 7f48d77fe640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48c00a3350 0x7f48c00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48c800bb60 tx=0x7f48c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.966+0000 7f48d7fff640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] conn(0x7f48c00acba0 0x7f48c00aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.966+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== osd.6 v2:172.21.15.31:6818/515354645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48cc004470 con 0x7f48c00acba0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.980+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 --> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48d8062d80 con 0x7f48c00acba0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.980+0000 7f48d4ff9640 1 -- 172.21.15.31:0/2251696035 <== osd.6 v2:172.21.15.31:6818/515354645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48d8062d80 con 0x7f48c00acba0 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] conn(0x7f48c00acba0 msgr2=0x7f48c00aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] conn(0x7f48c00acba0 0x7f48c00aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48c00a3350 msgr2=0x7f48c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f48c00a3350 0x7f48c00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48c800bb60 tx=0x7f48c800b390 comp rx=0 tx=0).stop 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 msgr2=0x7f48d8199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 --2- 172.21.15.31:0/2251696035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8101420 0x7f48d8199700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d950 tx=0x7f48cc00de10 comp rx=0 tx=0).stop 2024-09-15T23:58:54.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48d7fff640 1 -- 172.21.15.31:0/2251696035 reap_dead start 2024-09-15T23:58:54.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 shutdown_connections 2024-09-15T23:58:54.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 >> 172.21.15.31:0/2251696035 conn(0x7f48d80fc7a0 msgr2=0x7f48d80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:54.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 shutdown_connections 2024-09-15T23:58:54.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:52.981+0000 7f48de0a8640 1 -- 172.21.15.31:0/2251696035 wait complete. 2024-09-15T23:58:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:55 smithi031 ceph-mon[31203]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-15T23:58:55.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:55 smithi081 ceph-mon[35060]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-15T23:58:55.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:55 smithi102 ceph-mon[34458]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-15T23:58:55.588 INFO:teuthology.orchestra.run.smithi031.stdout:184683593776 2024-09-15T23:58:55.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 -- 172.21.15.31:0/2055396389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844101430 msgr2=0x7f284410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 --2- 172.21.15.31:0/2055396389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844101430 0x7f284410d960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f283c009cd0 tx=0x7f283c02a8e0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 -- 172.21.15.31:0/2055396389 shutdown_connections 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 --2- 172.21.15.31:0/2055396389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844101430 0x7f284410d960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 --2- 172.21.15.31:0/2055396389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2844100a90 0x7f2844100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 --2- 172.21.15.31:0/2055396389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 0x7f2844106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 -- 172.21.15.31:0/2055396389 >> 172.21.15.31:0/2055396389 conn(0x7f28440fc7d0 msgr2=0x7f28440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 -- 172.21.15.31:0/2055396389 shutdown_connections 2024-09-15T23:58:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.795+0000 7f2848d29640 1 -- 172.21.15.31:0/2055396389 wait complete. 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 Processor -- start 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 -- start start 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 0x7f28440fe820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2844101430 0x7f2844073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 0x7f2844073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f284406c4d0 con 0x7f2844100a90 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f284406c350 con 0x7f2844106a90 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f2848d29640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f284406c650 con 0x7f2844101430 2024-09-15T23:58:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f284259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 0x7f28440fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.796+0000 7f284259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 0x7f28440fe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58806/0 (socket says 172.21.15.31:58806) 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 -- 172.21.15.31:0/510332761 learned_addr learned my addr 172.21.15.31:0/510332761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f2841d9b640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2844101430 0x7f2844073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f2842d9d640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 0x7f2844073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 -- 172.21.15.31:0/510332761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2844101430 msgr2=0x7f2844073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2844101430 0x7f2844073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 -- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 msgr2=0x7f2844073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 0x7f2844073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2844073dd0 con 0x7f2844100a90 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f2842d9d640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2844106a90 0x7f2844073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.797+0000 7f284259c640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 0x7f28440fe820 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f283400c890 tx=0x7f283400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.798+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834004220 con 0x7f2844100a90 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.798+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28340043c0 con 0x7f2844100a90 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.798+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441af2c0 con 0x7f2844100a90 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.798+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441af800 con 0x7f2844100a90 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.798+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834004a00 con 0x7f2844100a90 2024-09-15T23:58:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.799+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2824001030 con 0x7f2844100a90 2024-09-15T23:58:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.801+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f28340079b0 con 0x7f2844100a90 2024-09-15T23:58:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.801+0000 7f28337fe640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f282c0a3590 0x7f282c0a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f28340047a0 con 0x7f2844100a90 2024-09-15T23:58:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f2841d9b640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f282c0a3590 0x7f282c0a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f28337fe640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] conn(0x7f282c0acde0 0x7f282c0af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f282c0af8b0 con 0x7f282c0acde0 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f28340cfa90 con 0x7f2844100a90 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f2841d9b640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f282c0a3590 0x7f282c0a5a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2838005fd0 tx=0x7f2838005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.802+0000 7f2842d9d640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] conn(0x7f282c0acde0 0x7f282c0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:55.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.803+0000 7f2842d9d640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] conn(0x7f282c0acde0 0x7f282c0af200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.803+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== osd.1 v2:172.21.15.81:6800/796455995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f282c0af8b0 con 0x7f282c0acde0 2024-09-15T23:58:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.826+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 --> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2824002d50 con 0x7f282c0acde0 2024-09-15T23:58:55.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f28337fe640 1 -- 172.21.15.31:0/510332761 <== osd.1 v2:172.21.15.81:6800/796455995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2824002d50 con 0x7f282c0acde0 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] conn(0x7f282c0acde0 msgr2=0x7f282c0af200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] conn(0x7f282c0acde0 0x7f282c0af200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f282c0a3590 msgr2=0x7f282c0a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f282c0a3590 0x7f282c0a5a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2838005fd0 tx=0x7f2838005e20 comp rx=0 tx=0).stop 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 msgr2=0x7f28440fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 --2- 172.21.15.31:0/510332761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2844100a90 0x7f28440fe820 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f283400c890 tx=0x7f283400cd50 comp rx=0 tx=0).stop 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2842d9d640 1 -- 172.21.15.31:0/510332761 reap_dead start 2024-09-15T23:58:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 shutdown_connections 2024-09-15T23:58:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 >> 172.21.15.31:0/510332761 conn(0x7f28440fc7d0 msgr2=0x7f284406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 shutdown_connections 2024-09-15T23:58:55.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:55.827+0000 7f2848d29640 1 -- 172.21.15.31:0/510332761 wait complete. 2024-09-15T23:58:56.261 INFO:teuthology.orchestra.run.smithi031.stdout:287762808858 2024-09-15T23:58:56.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-15T23:58:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:57 smithi031 ceph-mon[31203]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:57.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:57 smithi081 ceph-mon[35060]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:57.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:57 smithi102 ceph-mon[34458]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:57.639 INFO:teuthology.orchestra.run.smithi031.stdout:55834574922 2024-09-15T23:58:57.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-15T23:58:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:57.792 INFO:teuthology.orchestra.run.smithi031.stdout:236223201318 2024-09-15T23:58:57.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-15T23:58:58.396 INFO:teuthology.orchestra.run.smithi031.stdout:103079215167 2024-09-15T23:58:58.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-15T23:58:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.888+0000 7fb09cc44640 1 -- 172.21.15.31:0/383752930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 msgr2=0x7fb0981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.888+0000 7fb09cc44640 1 --2- 172.21.15.31:0/383752930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb0981029b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb080009900 tx=0x7fb08002a480 comp rx=0 tx=0).stop 2024-09-15T23:58:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 -- 172.21.15.31:0/383752930 shutdown_connections 2024-09-15T23:58:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 --2- 172.21.15.31:0/383752930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098102ef0 0x7fb09810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 --2- 172.21.15.31:0/383752930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb0981029b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 --2- 172.21.15.31:0/383752930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb098100e20 0x7fb098101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 -- 172.21.15.31:0/383752930 >> 172.21.15.31:0/383752930 conn(0x7fb0980fc7d0 msgr2=0x7fb0980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 -- 172.21.15.31:0/383752930 shutdown_connections 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 -- 172.21.15.31:0/383752930 wait complete. 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.889+0000 7fb09cc44640 1 Processor -- start 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 -- start start 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098100e20 0x7fb0981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb098198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb098102ef0 0x7fb09819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb098111fd0 con 0x7fb098102570 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb098111e50 con 0x7fb098102ef0 2024-09-15T23:58:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09cc44640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb098112150 con 0x7fb098100e20 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb098198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb098198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58828/0 (socket says 172.21.15.31:58828) 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 -- 172.21.15.31:0/2508952679 learned_addr learned my addr 172.21.15.31:0/2508952679 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb096d9d640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb098102ef0 0x7fb09819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09659c640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098100e20 0x7fb0981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 -- 172.21.15.31:0/2508952679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098100e20 msgr2=0x7fb0981984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098100e20 0x7fb0981984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 -- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb098102ef0 msgr2=0x7fb09819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb098102ef0 0x7fb09819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09819d480 con 0x7fb098102570 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb09659c640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb098100e20 0x7fb0981984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.890+0000 7fb08dd9b640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb098198a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb080006010 tx=0x7fb0800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08003f070 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09819d710 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb080003aa0 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080035030 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981a5050 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.891+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb098062980 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.893+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb080042070 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.893+0000 7fb08ffff640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb0840a3350 0x7fb0840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb09659c640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb0840a3350 0x7fb0840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb0800f0960 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb08ffff640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] conn(0x7fb0840acba0 0x7fb0840aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb080038e90 con 0x7fb0840acba0 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb0800f0e40 con 0x7fb098102570 2024-09-15T23:58:58.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb09659c640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb0840a3350 0x7fb0840a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb096d9d640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] conn(0x7fb0840acba0 0x7fb0840aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.894+0000 7fb096d9d640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] conn(0x7fb0840acba0 0x7fb0840aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.895+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== osd.4 v2:172.21.15.81:6808/3997776531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb080038e90 con 0x7fb0840acba0 2024-09-15T23:58:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.909+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 --> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb098062d80 con 0x7fb0840acba0 2024-09-15T23:58:58.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb08ffff640 1 -- 172.21.15.31:0/2508952679 <== osd.4 v2:172.21.15.81:6808/3997776531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb098062d80 con 0x7fb0840acba0 2024-09-15T23:58:58.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] conn(0x7fb0840acba0 msgr2=0x7fb0840aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] conn(0x7fb0840acba0 0x7fb0840aefc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb0840a3350 msgr2=0x7fb0840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb0840a3350 0x7fb0840a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).stop 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 msgr2=0x7fb098198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 --2- 172.21.15.31:0/2508952679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb098102570 0x7fb098198a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb080006010 tx=0x7fb0800042f0 comp rx=0 tx=0).stop 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb096d9d640 1 -- 172.21.15.31:0/2508952679 reap_dead start 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 shutdown_connections 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 >> 172.21.15.31:0/2508952679 conn(0x7fb0980fc7d0 msgr2=0x7fb0980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 shutdown_connections 2024-09-15T23:58:58.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:58.910+0000 7fb09cc44640 1 -- 172.21.15.31:0/2508952679 wait complete. 2024-09-15T23:58:59.194 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:58:59 smithi031 ceph-mon[31203]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:59.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:58:59 smithi081 ceph-mon[35060]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:59.314 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:58:59.354 INFO:teuthology.orchestra.run.smithi031.stdout:34359738447 2024-09-15T23:58:59.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-15T23:58:59.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:58:59 smithi102 ceph-mon[34458]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 -- 172.21.15.31:0/591717252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 msgr2=0x7f1f3c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 --2- 172.21.15.31:0/591717252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c10bc00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009900 tx=0x7f1f3002a410 comp rx=0 tx=0).stop 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 -- 172.21.15.31:0/591717252 shutdown_connections 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 --2- 172.21.15.31:0/591717252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c10c2f0 0x7f1f3c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 --2- 172.21.15.31:0/591717252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c10bc00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 --2- 172.21.15.31:0/591717252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f3c102df0 0x7f1f3c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 -- 172.21.15.31:0/591717252 >> 172.21.15.31:0/591717252 conn(0x7f1f3c0fc7d0 msgr2=0x7f1f3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 -- 172.21.15.31:0/591717252 shutdown_connections 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.852+0000 7f1f43f00640 1 -- 172.21.15.31:0/591717252 wait complete. 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 Processor -- start 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 -- start start 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c102df0 0x7f1f3c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f3c10c2f0 0x7f1f3c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111fe0 con 0x7f1f3c1037a0 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111e60 con 0x7f1f3c102df0 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f43f00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c112160 con 0x7f1f3c10c2f0 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58854/0 (socket says 172.21.15.31:58854) 2024-09-15T23:58:59.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 -- 172.21.15.31:0/193733849 learned_addr learned my addr 172.21.15.31:0/193733849 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4249d640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f3c10c2f0 0x7f1f3c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f41c9c640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c102df0 0x7f1f3c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 -- 172.21.15.31:0/193733849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f3c10c2f0 msgr2=0x7f1f3c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f3c10c2f0 0x7f1f3c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 -- 172.21.15.31:0/193733849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c102df0 msgr2=0x7f1f3c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c102df0 0x7f1f3c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f4149b640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c19d540 con 0x7f1f3c1037a0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.853+0000 7f1f41c9c640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f3c102df0 0x7f1f3c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f4149b640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c198ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002a930 tx=0x7f1f30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3003f030 con 0x7f1f3c1037a0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c19d7d0 con 0x7f1f3c1037a0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30004490 con 0x7f1f3c1037a0 2024-09-15T23:58:59.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30035070 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.854+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1a5110 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.855+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f3c062900 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.856+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1f30042030 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f3affd640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f240a32d0 0x7f1f240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f41c9c640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f240a32d0 0x7f1f240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1f300f02b0 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f3affd640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] conn(0x7f1f240acb20 0x7f1f240aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f240af5f0 con 0x7f1f240acb20 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f1f300f0790 con 0x7f1f3c1037a0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f4249d640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] conn(0x7f1f240acb20 0x7f1f240aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f41c9c640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f240a32d0 0x7f1f240a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c007e70 tx=0x7f1f2c007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.857+0000 7f1f4249d640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] conn(0x7f1f240acb20 0x7f1f240aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:58:59.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.858+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== osd.9 v2:172.21.15.31:6826/1878664513 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f240af5f0 con 0x7f1f240acb20 2024-09-15T23:58:59.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.872+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 --> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f3c062d00 con 0x7f1f240acb20 2024-09-15T23:58:59.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f3affd640 1 -- 172.21.15.31:0/193733849 <== osd.9 v2:172.21.15.31:6826/1878664513 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f3c062d00 con 0x7f1f240acb20 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] conn(0x7f1f240acb20 msgr2=0x7f1f240aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] conn(0x7f1f240acb20 0x7f1f240aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f240a32d0 msgr2=0x7f1f240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f240a32d0 0x7f1f240a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c007e70 tx=0x7f1f2c007850 comp rx=0 tx=0).stop 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 msgr2=0x7f1f3c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:58:59.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 --2- 172.21.15.31:0/193733849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f3c1037a0 0x7f1f3c198ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002a930 tx=0x7f1f30004270 comp rx=0 tx=0).stop 2024-09-15T23:58:59.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f4249d640 1 -- 172.21.15.31:0/193733849 reap_dead start 2024-09-15T23:58:59.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 shutdown_connections 2024-09-15T23:58:59.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 >> 172.21.15.31:0/193733849 conn(0x7f1f3c0fc7d0 msgr2=0x7f1f3c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:58:59.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 shutdown_connections 2024-09-15T23:58:59.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:58:59.873+0000 7f1f43f00640 1 -- 172.21.15.31:0/193733849 wait complete. 2024-09-15T23:59:00.583 INFO:teuthology.orchestra.run.smithi031.stdout:266287972386 2024-09-15T23:59:00.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.025+0000 7f3df39ee640 1 -- 172.21.15.31:0/353508818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec102d30 msgr2=0x7f3dec103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.025+0000 7f3df39ee640 1 --2- 172.21.15.31:0/353508818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec102d30 0x7f3dec103110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009930 tx=0x7f3de802a420 comp rx=0 tx=0).stop 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 -- 172.21.15.31:0/353508818 shutdown_connections 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 --2- 172.21.15.31:0/353508818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dec1082a0 0x7f3dec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 --2- 172.21.15.31:0/353508818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dec1036e0 0x7f3dec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 --2- 172.21.15.31:0/353508818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec102d30 0x7f3dec103110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 -- 172.21.15.31:0/353508818 >> 172.21.15.31:0/353508818 conn(0x7f3dec0fc7d0 msgr2=0x7f3dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 -- 172.21.15.31:0/353508818 shutdown_connections 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 -- 172.21.15.31:0/353508818 wait complete. 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 Processor -- start 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.026+0000 7f3df39ee640 1 -- start start 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dec102d30 0x7f3dec1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 0x7f3dec198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dec1082a0 0x7f3dec19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec111f50 con 0x7f3dec1036e0 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec111dd0 con 0x7f3dec102d30 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec1120d0 con 0x7f3dec1082a0 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 0x7f3dec198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 0x7f3dec198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58876/0 (socket says 172.21.15.31:58876) 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 -- 172.21.15.31:0/3271106077 learned_addr learned my addr 172.21.15.31:0/3271106077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df178a640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dec102d30 0x7f3dec1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df1f8b640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dec1082a0 0x7f3dec19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 -- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dec1082a0 msgr2=0x7f3dec19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dec1082a0 0x7f3dec19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 -- 172.21.15.31:0/3271106077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dec102d30 msgr2=0x7f3dec1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dec102d30 0x7f3dec1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec19d480 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df0f89640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 0x7f3dec198a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d740 tx=0x7f3ddc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc017070 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ddc00de50 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec19d770 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.027+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00f620 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.028+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec1a5050 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.028+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dec062980 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.030+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3ddc01a070 con 0x7f3dec1036e0 2024-09-15T23:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.030+0000 7f3de27fc640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3dd80a32d0 0x7f3dd80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.030+0000 7f3df178a640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3dd80a32d0 0x7f3dd80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.030+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3ddc0cf6e0 con 0x7f3dec1036e0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.031+0000 7f3de27fc640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] conn(0x7f3dd80acb20 0x7f3dd80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.031+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ddc01de50 con 0x7f3dd80acb20 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.031+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc0cfbc0 con 0x7f3dec1036e0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.031+0000 7f3df178a640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3dd80a32d0 0x7f3dd80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009900 tx=0x7f3de8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.031+0000 7f3df1f8b640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] conn(0x7f3dd80acb20 0x7f3dd80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.032+0000 7f3df1f8b640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] conn(0x7f3dd80acb20 0x7f3dd80aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.032+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== osd.11 v2:172.21.15.102:6824/4272602493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ddc01de50 con 0x7f3dd80acb20 2024-09-15T23:59:01.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.046+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 --> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3dec062e00 con 0x7f3dd80acb20 2024-09-15T23:59:01.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3de27fc640 1 -- 172.21.15.31:0/3271106077 <== osd.11 v2:172.21.15.102:6824/4272602493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3dec062e00 con 0x7f3dd80acb20 2024-09-15T23:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] conn(0x7f3dd80acb20 msgr2=0x7f3dd80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] conn(0x7f3dd80acb20 0x7f3dd80aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3dd80a32d0 msgr2=0x7f3dd80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f3dd80a32d0 0x7f3dd80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009900 tx=0x7f3de8000bc0 comp rx=0 tx=0).stop 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 msgr2=0x7f3dec198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 --2- 172.21.15.31:0/3271106077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dec1036e0 0x7f3dec198a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d740 tx=0x7f3ddc00dc00 comp rx=0 tx=0).stop 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df1f8b640 1 -- 172.21.15.31:0/3271106077 reap_dead start 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 shutdown_connections 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 >> 172.21.15.31:0/3271106077 conn(0x7f3dec0fc7d0 msgr2=0x7f3dec105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 shutdown_connections 2024-09-15T23:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:01.047+0000 7f3df39ee640 1 -- 172.21.15.31:0/3271106077 wait complete. 2024-09-15T23:59:01.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:01 smithi031 ceph-mon[31203]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T23:59:01.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:01 smithi081 ceph-mon[35060]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T23:59:01.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:01 smithi102 ceph-mon[34458]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T23:59:01.699 INFO:teuthology.orchestra.run.smithi031.stdout:133143986234 2024-09-15T23:59:01.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-15T23:59:01.904 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:02.444 INFO:teuthology.orchestra.run.smithi031.stdout:317827579928 2024-09-15T23:59:02.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-15T23:59:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- 172.21.15.31:0/733809291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4100a80 msgr2=0x7f49d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/733809291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4100a80 0x7f49d4100ee0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009930 tx=0x7f49c802a420 comp rx=0 tx=0).stop 2024-09-15T23:59:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- 172.21.15.31:0/733809291 shutdown_connections 2024-09-15T23:59:02.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/733809291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d4101420 0x7f49d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/733809291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4100a80 0x7f49d4100ee0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/733809291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49d4106a80 0x7f49d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- 172.21.15.31:0/733809291 >> 172.21.15.31:0/733809291 conn(0x7f49d40fc7a0 msgr2=0x7f49d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- 172.21.15.31:0/733809291 shutdown_connections 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- 172.21.15.31:0/733809291 wait complete. 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 Processor -- start 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.115+0000 7f49dc2d9640 1 -- start start 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d4100a80 0x7f49d4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49d4101420 0x7f49d41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 0x7f49d41a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110d30 con 0x7f49d4106a80 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110bb0 con 0x7f49d4101420 2024-09-15T23:59:02.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4110eb0 con 0x7f49d4100a80 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 0x7f49d41a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 0x7f49d41a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58210/0 (socket says 172.21.15.31:58210) 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 -- 172.21.15.31:0/428875412 learned_addr learned my addr 172.21.15.31:0/428875412 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da075640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d4100a80 0x7f49d4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49d9874640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49d4101420 0x7f49d41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 -- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d4100a80 msgr2=0x7f49d4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d4100a80 0x7f49d4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 -- 172.21.15.31:0/428875412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49d4101420 msgr2=0x7f49d41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49d4101420 0x7f49d41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41a4740 con 0x7f49d4106a80 2024-09-15T23:59:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49da876640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 0x7f49d41a4200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c970 tx=0x7f49cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc007bb0 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc007d50 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41a4910 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc018700 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.116+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a4dd0 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.117+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49d4062900 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.119+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f49cc00fa20 con 0x7f49d4106a80 2024-09-15T23:59:02.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.119+0000 7f49d37fe640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49bc0a3350 0x7f49bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49da075640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49bc0a3350 0x7f49bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f49cc0cfb70 con 0x7f49d4106a80 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d37fe640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] conn(0x7f49bc0acba0 0x7f49bc0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49cc01ee50 con 0x7f49bc0acba0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f49cc0d0050 con 0x7f49d4106a80 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49da075640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49bc0a3350 0x7f49bc0a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005c40 tx=0x7f49c400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d9874640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] conn(0x7f49bc0acba0 0x7f49bc0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.120+0000 7f49d9874640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] conn(0x7f49bc0acba0 0x7f49bc0aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.121+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== osd.5 v2:172.21.15.102:6808/3259467463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49cc01ee50 con 0x7f49bc0acba0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 --> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49d4062d80 con 0x7f49bc0acba0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49d37fe640 1 -- 172.21.15.31:0/428875412 <== osd.5 v2:172.21.15.102:6808/3259467463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49d4062d80 con 0x7f49bc0acba0 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] conn(0x7f49bc0acba0 msgr2=0x7f49bc0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] conn(0x7f49bc0acba0 0x7f49bc0aefc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49bc0a3350 msgr2=0x7f49bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f49bc0a3350 0x7f49bc0a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005c40 tx=0x7f49c400a590 comp rx=0 tx=0).stop 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 msgr2=0x7f49d41a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49dc2d9640 1 --2- 172.21.15.31:0/428875412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4106a80 0x7f49d41a4200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c970 tx=0x7f49cc00ce30 comp rx=0 tx=0).stop 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.137+0000 7f49da876640 1 -- 172.21.15.31:0/428875412 reap_dead start 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.138+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 shutdown_connections 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.138+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 >> 172.21.15.31:0/428875412 conn(0x7f49d40fc7a0 msgr2=0x7f49d40fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.138+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 shutdown_connections 2024-09-15T23:59:02.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:02.138+0000 7f49dc2d9640 1 -- 172.21.15.31:0/428875412 wait complete. 2024-09-15T23:59:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:03 smithi031 ceph-mon[31203]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-15T23:59:03.295 INFO:teuthology.orchestra.run.smithi031.stdout:154618822710 2024-09-15T23:59:03.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-15T23:59:03.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:03 smithi081 ceph-mon[35060]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-15T23:59:03.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:03 smithi102 ceph-mon[34458]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-15T23:59:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.088+0000 7f2948467640 1 -- 172.21.15.31:0/3215298913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102ef0 msgr2=0x7f294010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.088+0000 7f2948467640 1 --2- 172.21.15.31:0/3215298913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102ef0 0x7f294010da40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2938009be0 tx=0x7f293802a750 comp rx=0 tx=0).stop 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 -- 172.21.15.31:0/3215298913 shutdown_connections 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 --2- 172.21.15.31:0/3215298913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102ef0 0x7f294010da40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 --2- 172.21.15.31:0/3215298913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2940102570 0x7f29401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 --2- 172.21.15.31:0/3215298913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 0x7f2940101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 -- 172.21.15.31:0/3215298913 >> 172.21.15.31:0/3215298913 conn(0x7f29400fc7d0 msgr2=0x7f29400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 -- 172.21.15.31:0/3215298913 shutdown_connections 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 -- 172.21.15.31:0/3215298913 wait complete. 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.089+0000 7f2948467640 1 Processor -- start 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 -- start start 2024-09-15T23:59:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 0x7f294019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2940102ef0 0x7f294019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2940110e30 con 0x7f2940102570 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2940110cb0 con 0x7f2940100e20 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2948467640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2940110fb0 con 0x7f2940102ef0 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58242/0 (socket says 172.21.15.31:58242) 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 -- 172.21.15.31:0/3386738859 learned_addr learned my addr 172.21.15.31:0/3386738859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2946203640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 0x7f294019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 -- 172.21.15.31:0/3386738859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2940102ef0 msgr2=0x7f294019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2940102ef0 0x7f294019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 -- 172.21.15.31:0/3386738859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 msgr2=0x7f294019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 0x7f294019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.090+0000 7f2945a02640 1 -- 172.21.15.31:0/3386738859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019f5d0 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f2945a02640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f293400e9f0 tx=0x7f293400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293400cd80 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294019f8c0 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29340054d0 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934010490 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.091+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a71a0 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.092+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2940101200 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.093+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2934005020 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.094+0000 7f29337fe640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 0x7f292c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.094+0000 7f2946203640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 0x7f292c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.094+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f29340cfc90 con 0x7f2940102570 2024-09-15T23:59:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.095+0000 7f2946203640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 0x7f292c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f293c009530 tx=0x7f293c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.098+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f293409ca00 con 0x7f2940102570 2024-09-15T23:59:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 -- 172.21.15.31:0/120559738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa0102d40 msgr2=0x7f2fa0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 --2- 172.21.15.31:0/120559738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa0102d40 0x7f2fa0103120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009900 tx=0x7f2f9002a480 comp rx=0 tx=0).stop 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 -- 172.21.15.31:0/120559738 shutdown_connections 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 --2- 172.21.15.31:0/120559738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa010c170 0x7f2fa010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 --2- 172.21.15.31:0/120559738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0103660 0x7f2fa010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 --2- 172.21.15.31:0/120559738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa0102d40 0x7f2fa0103120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 -- 172.21.15.31:0/120559738 >> 172.21.15.31:0/120559738 conn(0x7f2fa00fc7a0 msgr2=0x7f2fa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 -- 172.21.15.31:0/120559738 shutdown_connections 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.971+0000 7f2fa4876640 1 -- 172.21.15.31:0/120559738 wait complete. 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 Processor -- start 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 -- start start 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 0x7f2fa01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 0x7f2fa01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0111eb0 con 0x7f2fa010c170 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0111d30 con 0x7f2fa0102d40 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2fa4876640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0112030 con 0x7f2fa0103660 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2f9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2f9ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58226/0 (socket says 172.21.15.31:58226) 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.972+0000 7f2f9ed9d640 1 -- 172.21.15.31:0/3977933436 learned_addr learned my addr 172.21.15.31:0/3977933436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 -- 172.21.15.31:0/3977933436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 msgr2=0x7f2fa01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9dd9b640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 0x7f2fa01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9e59c640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 0x7f2fa01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 0x7f2fa01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 -- 172.21.15.31:0/3977933436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 msgr2=0x7f2fa01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 0x7f2fa01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 -- 172.21.15.31:0/3977933436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa019d460 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9dd9b640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 0x7f2fa01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9e59c640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 0x7f2fa01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f9ed9d640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800cad0 tx=0x7f2f9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800ae20 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f98005020 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa019d750 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800f6b0 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.973+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01a5030 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.974+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa0103120 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.975+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2f980051d0 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.976+0000 7f2f8f7fe640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 0x7f2f880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.976+0000 7f2f9e59c640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 0x7f2f880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.976+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2f980cfa50 con 0x7f2fa010c170 2024-09-15T23:59:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.976+0000 7f2f9e59c640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 0x7f2f880a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005ab0 tx=0x7f2f90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stdout:184683593778 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:03.980+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f98014030 con 0x7f2fa010c170 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.181+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2fa0106c80 con 0x7f2fa010c170 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.182+0000 7f2f8f7fe640 1 -- 172.21.15.31:0/3977933436 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2f9809c7c0 con 0x7f2fa010c170 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 msgr2=0x7f2f880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 0x7f2f880a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005ab0 tx=0x7f2f90000bc0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 msgr2=0x7f2fa019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800cad0 tx=0x7f2f9800cf90 comp rx=0 tx=0).stop 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 shutdown_connections 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f880a32d0 0x7f2f880a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa010c170 0x7f2fa019cd80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0103660 0x7f2fa01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 --2- 172.21.15.31:0/3977933436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa0102d40 0x7f2fa01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 >> 172.21.15.31:0/3977933436 conn(0x7f2fa00fc7a0 msgr2=0x7f2fa00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 shutdown_connections 2024-09-15T23:59:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.184+0000 7f2fa4876640 1 -- 172.21.15.31:0/3977933436 wait complete. 2024-09-15T23:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.285+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f29401a7400 con 0x7f2940102570 2024-09-15T23:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.285+0000 7f29337fe640 1 -- 172.21.15.31:0/3386738859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f293409c300 con 0x7f2940102570 2024-09-15T23:59:04.286 INFO:teuthology.orchestra.run.smithi031.stdout:206158430252 2024-09-15T23:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 msgr2=0x7f292c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 0x7f292c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f293c009530 tx=0x7f293c009260 comp rx=0 tx=0).stop 2024-09-15T23:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 msgr2=0x7f294019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f293400e9f0 tx=0x7f293400edf0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 shutdown_connections 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f292c0a32d0 0x7f292c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2940102ef0 0x7f294019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2940102570 0x7f294019ab60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 --2- 172.21.15.31:0/3386738859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2940100e20 0x7f294019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 >> 172.21.15.31:0/3386738859 conn(0x7f29400fc7d0 msgr2=0x7f294006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 shutdown_connections 2024-09-15T23:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:04.287+0000 7f2948467640 1 -- 172.21.15.31:0/3386738859 wait complete. 2024-09-15T23:59:05.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:05 smithi031 ceph-mon[31203]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-15T23:59:05.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:05 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3977933436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T23:59:05.182 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:05 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3386738859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T23:59:05.212 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593776 got 184683593778 for osd.6 2024-09-15T23:59:05.212 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:05 smithi081 ceph-mon[35060]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-15T23:59:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:05 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3977933436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T23:59:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:05 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3386738859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T23:59:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:05 smithi102 ceph-mon[34458]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-15T23:59:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:05 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3977933436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T23:59:05.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:05 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3386738859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 -- 172.21.15.31:0/3324882576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 msgr2=0x7fcf1c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 --2- 172.21.15.31:0/3324882576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c10ec50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009cd0 tx=0x7fcf1402a8e0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 -- 172.21.15.31:0/3324882576 shutdown_connections 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 --2- 172.21.15.31:0/3324882576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c10ec50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 --2- 172.21.15.31:0/3324882576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 --2- 172.21.15.31:0/3324882576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 0x7fcf1c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 -- 172.21.15.31:0/3324882576 >> 172.21.15.31:0/3324882576 conn(0x7fcf1c0fc7d0 msgr2=0x7fcf1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 -- 172.21.15.31:0/3324882576 shutdown_connections 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 -- 172.21.15.31:0/3324882576 wait complete. 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.617+0000 7fcf2438d640 1 Processor -- start 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 -- start start 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 0x7fcf1c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c111f60 con 0x7fcf1c1082e0 2024-09-15T23:59:05.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c111de0 con 0x7fcf1c102db0 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2438d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1120e0 con 0x7fcf1c103760 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58254/0 (socket says 172.21.15.31:58254) 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 -- 172.21.15.31:0/1691086491 learned_addr learned my addr 172.21.15.31:0/1691086491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf22129640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 0x7fcf1c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf21928640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 -- 172.21.15.31:0/1691086491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 msgr2=0x7fcf1c100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 -- 172.21.15.31:0/1691086491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 msgr2=0x7fcf1c100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:05.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 0x7fcf1c100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 -- 172.21.15.31:0/1691086491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c1a6ad0 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf21928640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.618+0000 7fcf2292a640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf140097b0 tx=0x7fcf14003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.619+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14004240 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.619+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf140305a0 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.619+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c1a6d60 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.619+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14045650 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.619+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c1a72a0 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.620+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c103190 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.621+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcf14043020 con 0x7fcf1c1082e0 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.622+0000 7fcf0f7fe640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 0x7fcf080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:05.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.622+0000 7fcf22129640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 0x7fcf080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.622+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcf1402e030 con 0x7fcf1c1082e0 2024-09-15T23:59:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.622+0000 7fcf22129640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 0x7fcf080a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf180058c0 tx=0x7fcf1800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:05.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.626+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf1404a060 con 0x7fcf1c1082e0 2024-09-15T23:59:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.804+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcf1c10a2c0 con 0x7fcf1c1082e0 2024-09-15T23:59:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.805+0000 7fcf0f7fe640 1 -- 172.21.15.31:0/1691086491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf140bd610 con 0x7fcf1c1082e0 2024-09-15T23:59:05.806 INFO:teuthology.orchestra.run.smithi031.stdout:77309411398 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 msgr2=0x7fcf080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 0x7fcf080a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf180058c0 tx=0x7fcf1800a2d0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 msgr2=0x7fcf1c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf140097b0 tx=0x7fcf14003ba0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 shutdown_connections 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcf080a32d0 0x7fcf080a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf1c1082e0 0x7fcf1c101490 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf1c103760 0x7fcf1c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 --2- 172.21.15.31:0/1691086491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf1c102db0 0x7fcf1c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 >> 172.21.15.31:0/1691086491 conn(0x7fcf1c0fc7d0 msgr2=0x7fcf1c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 shutdown_connections 2024-09-15T23:59:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:05.807+0000 7fcf2438d640 1 -- 172.21.15.31:0/1691086491 wait complete. 2024-09-15T23:59:06.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:06 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1691086491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T23:59:06.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:06 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1691086491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T23:59:06.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:06 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1691086491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T23:59:06.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430249 got 206158430252 for osd.7 2024-09-15T23:59:06.634 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:06.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 -- 172.21.15.31:0/737722312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 msgr2=0x7fd90410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 --2- 172.21.15.31:0/737722312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd90410d950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009cd0 tx=0x7fd8fc02a8e0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 -- 172.21.15.31:0/737722312 shutdown_connections 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 --2- 172.21.15.31:0/737722312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd90410d950 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 --2- 172.21.15.31:0/737722312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 0x7fd904100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 --2- 172.21.15.31:0/737722312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 0x7fd904106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 -- 172.21.15.31:0/737722312 >> 172.21.15.31:0/737722312 conn(0x7fd9040fc7a0 msgr2=0x7fd9040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 -- 172.21.15.31:0/737722312 shutdown_connections 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.901+0000 7fd90b976640 1 -- 172.21.15.31:0/737722312 wait complete. 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 Processor -- start 2024-09-15T23:59:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 -- start start 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 0x7fd904102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 0x7fd9041030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd904110d30 con 0x7fd904101420 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd904110bb0 con 0x7fd904106a80 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd90b976640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd904110eb0 con 0x7fd904100a80 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.902+0000 7fd908f11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58282/0 (socket says 172.21.15.31:58282) 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 -- 172.21.15.31:0/4160891915 learned_addr learned my addr 172.21.15.31:0/4160891915 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd909712640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 0x7fd904102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd909f13640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 0x7fd9041030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 -- 172.21.15.31:0/4160891915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 msgr2=0x7fd904102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 0x7fd904102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 -- 172.21.15.31:0/4160891915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 msgr2=0x7fd9041030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 0x7fd9041030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 -- 172.21.15.31:0/4160891915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041a6880 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd908f11640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d950 tx=0x7fd8f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4017070 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041a6b70 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8f4009c30 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400f620 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.903+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041a70b0 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.904+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd904100ee0 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.905+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd8f40109a0 con 0x7fd904101420 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.906+0000 7fd8fa7fc640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 0x7fd8f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.906+0000 7fd909712640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 0x7fd8f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.906+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd8f40cf470 con 0x7fd904101420 2024-09-15T23:59:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.907+0000 7fd909712640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 0x7fd8f00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd90000bb60 tx=0x7fd90000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:06.910+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8f4009de0 con 0x7fd904101420 2024-09-15T23:59:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.092+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd9041a7280 con 0x7fd904101420 2024-09-15T23:59:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.092+0000 7fd8fa7fc640 1 -- 172.21.15.31:0/4160891915 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd8f409bb90 con 0x7fd904101420 2024-09-15T23:59:07.093 INFO:teuthology.orchestra.run.smithi031.stdout:287762808861 2024-09-15T23:59:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 msgr2=0x7fd8f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 0x7fd8f00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd90000bb60 tx=0x7fd90000b390 comp rx=0 tx=0).stop 2024-09-15T23:59:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 msgr2=0x7fd904102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400d950 tx=0x7fd8f400de10 comp rx=0 tx=0).stop 2024-09-15T23:59:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 shutdown_connections 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd8f00a32d0 0x7fd8f00a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd904106a80 0x7fd9041030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd904101420 0x7fd904102b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 --2- 172.21.15.31:0/4160891915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd904100a80 0x7fd904102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 >> 172.21.15.31:0/4160891915 conn(0x7fd9040fc7a0 msgr2=0x7fd90406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.094+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 shutdown_connections 2024-09-15T23:59:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.095+0000 7fd90b976640 1 -- 172.21.15.31:0/4160891915 wait complete. 2024-09-15T23:59:07.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:07 smithi031 ceph-mon[31203]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-15T23:59:07.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:07 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:59:07.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:07 smithi081 ceph-mon[35060]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-15T23:59:07.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:07 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:59:07.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:07 smithi102 ceph-mon[34458]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-15T23:59:07.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:07 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:59:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 -- 172.21.15.31:0/1278264024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590106f10 msgr2=0x7f759010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 --2- 172.21.15.31:0/1278264024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590106f10 0x7f759010ea90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f758802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 -- 172.21.15.31:0/1278264024 shutdown_connections 2024-09-15T23:59:07.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 --2- 172.21.15.31:0/1278264024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590106f10 0x7f759010ea90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 --2- 172.21.15.31:0/1278264024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590102790 0x7f75901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 --2- 172.21.15.31:0/1278264024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590101de0 0x7f75901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 -- 172.21.15.31:0/1278264024 >> 172.21.15.31:0/1278264024 conn(0x7f75900fd8c0 msgr2=0x7f75900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.465+0000 7f759590d640 1 -- 172.21.15.31:0/1278264024 shutdown_connections 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 -- 172.21.15.31:0/1278264024 wait complete. 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 Processor -- start 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 -- start start 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 0x7f7590198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590106f10 0x7f759019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7590111e70 con 0x7f7590102790 2024-09-15T23:59:07.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7590111cf0 con 0x7f7590101de0 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.466+0000 7f759590d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7590111ff0 con 0x7f7590106f10 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f758effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 0x7f7590198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58312/0 (socket says 172.21.15.31:58312) 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 -- 172.21.15.31:0/2418850809 learned_addr learned my addr 172.21.15.31:0/2418850809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f758f7fe640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590106f10 0x7f759019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 -- 172.21.15.31:0/2418850809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590106f10 msgr2=0x7f759019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590106f10 0x7f759019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 -- 172.21.15.31:0/2418850809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 msgr2=0x7f7590198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 0x7f7590198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 -- 172.21.15.31:0/2418850809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759019d2e0 con 0x7f7590102790 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f758effd640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 0x7f7590198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f7587fff640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f757800e9f0 tx=0x7f757800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:07.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800cd80 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75780108e0 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759019d5d0 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578018920 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.467+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a4eb0 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.468+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75901021c0 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.470+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7578010a90 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.471+0000 7f759490b640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 0x7f757c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.471+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f75780cffe0 con 0x7f7590102790 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.471+0000 7f758effd640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 0x7f757c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.472+0000 7f758effd640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 0x7f757c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fc0 tx=0x7f7580005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:07.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.474+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f757809c700 con 0x7f7590102790 2024-09-15T23:59:07.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.650+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f759010a150 con 0x7f7590102790 2024-09-15T23:59:07.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.651+0000 7f759490b640 1 -- 172.21.15.31:0/2418850809 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75780a0a00 con 0x7f7590102790 2024-09-15T23:59:07.652 INFO:teuthology.orchestra.run.smithi031.stdout:55834574924 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 msgr2=0x7f757c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 0x7f757c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fc0 tx=0x7f7580005f90 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 msgr2=0x7f7590198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f757800e9f0 tx=0x7f757800eeb0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 shutdown_connections 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f757c0a32d0 0x7f757c0a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7590106f10 0x7f759019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7590102790 0x7f7590198870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 --2- 172.21.15.31:0/2418850809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7590101de0 0x7f7590198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:07.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 >> 172.21.15.31:0/2418850809 conn(0x7f75900fd8c0 msgr2=0x7f75900fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:07.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 shutdown_connections 2024-09-15T23:59:07.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:07.653+0000 7f759590d640 1 -- 172.21.15.31:0/2418850809 wait complete. 2024-09-15T23:59:08.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411398 for osd.2 2024-09-15T23:59:08.024 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:08 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4160891915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T23:59:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:08 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2418850809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T23:59:08.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:08 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4160891915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T23:59:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:08 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2418850809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T23:59:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:08 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4160891915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T23:59:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:08 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2418850809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T23:59:08.620 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808858 got 287762808861 for osd.10 2024-09-15T23:59:08.620 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:09.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:09 smithi031 ceph-mon[31203]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-15T23:59:09.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:09 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:09.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574924 for osd.1 2024-09-15T23:59:09.295 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:09.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:09 smithi081 ceph-mon[35060]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-15T23:59:09.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:09 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:09 smithi102 ceph-mon[34458]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-15T23:59:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:09 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 -- 172.21.15.31:0/1809597714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8106f10 msgr2=0x7f2db810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 --2- 172.21.15.31:0/1809597714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8106f10 0x7f2db810ea90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009ca0 tx=0x7f2db002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 -- 172.21.15.31:0/1809597714 shutdown_connections 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 --2- 172.21.15.31:0/1809597714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8106f10 0x7f2db810ea90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 --2- 172.21.15.31:0/1809597714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8102790 0x7f2db81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 --2- 172.21.15.31:0/1809597714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 0x7f2db81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 -- 172.21.15.31:0/1809597714 >> 172.21.15.31:0/1809597714 conn(0x7f2db80fd8c0 msgr2=0x7f2db80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 -- 172.21.15.31:0/1809597714 shutdown_connections 2024-09-15T23:59:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.173+0000 7f2dbde12640 1 -- 172.21.15.31:0/1809597714 wait complete. 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 Processor -- start 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 -- start start 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 0x7f2db8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8106f10 0x7f2db819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111e20 con 0x7f2db8102790 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111ca0 con 0x7f2db8106f10 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2dbde12640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2db8111fa0 con 0x7f2db8101de0 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2db6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.174+0000 7f2db6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58336/0 (socket says 172.21.15.31:58336) 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 -- 172.21.15.31:0/2521816613 learned_addr learned my addr 172.21.15.31:0/2521816613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db7fff640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8106f10 0x7f2db819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db77fe640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 0x7f2db8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 -- 172.21.15.31:0/2521816613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 msgr2=0x7f2db8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 0x7f2db8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 -- 172.21.15.31:0/2521816613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8106f10 msgr2=0x7f2db819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8106f10 0x7f2db819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 -- 172.21.15.31:0/2521816613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db819d310 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db6ffd640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c890 tx=0x7f2dac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac004220 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dac0043c0 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac004a00 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db819d600 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.175+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db81a4ee0 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.176+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db81021c0 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.178+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2dac004ba0 con 0x7f2db8102790 2024-09-15T23:59:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.178+0000 7f2db4ff9640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 0x7f2da00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.178+0000 7f2db77fe640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 0x7f2da00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.179+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2dac0cf4d0 con 0x7f2db8102790 2024-09-15T23:59:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.179+0000 7f2db77fe640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 0x7f2da00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2da800bbe0 tx=0x7f2da800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.183+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dac014030 con 0x7f2db8102790 2024-09-15T23:59:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 -- 172.21.15.31:0/3789509187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0106f20 msgr2=0x7f11e010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 --2- 172.21.15.31:0/3789509187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0106f20 0x7f11e010eaa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ca0 tx=0x7f11d802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 -- 172.21.15.31:0/3789509187 shutdown_connections 2024-09-15T23:59:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 --2- 172.21.15.31:0/3789509187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0106f20 0x7f11e010eaa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 --2- 172.21.15.31:0/3789509187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e01027a0 0x7f11e01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 --2- 172.21.15.31:0/3789509187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e0101df0 0x7f11e01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 -- 172.21.15.31:0/3789509187 >> 172.21.15.31:0/3789509187 conn(0x7f11e00fd8f0 msgr2=0x7f11e00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 -- 172.21.15.31:0/3789509187 shutdown_connections 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.348+0000 7f11e4816640 1 -- 172.21.15.31:0/3789509187 wait complete. 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 Processor -- start 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 -- start start 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e01027a0 0x7f11e0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 0x7f11e019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0111e30 con 0x7f11e0101df0 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0111cb0 con 0x7f11e01027a0 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11e4816640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0111fb0 con 0x7f11e0106f20 2024-09-15T23:59:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58356/0 (socket says 172.21.15.31:58356) 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 -- 172.21.15.31:0/4160443454 learned_addr learned my addr 172.21.15.31:0/4160443454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11ddd9b640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e01027a0 0x7f11e0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11ded9d640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 0x7f11e019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 -- 172.21.15.31:0/4160443454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 msgr2=0x7f11e019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 0x7f11e019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 -- 172.21.15.31:0/4160443454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e01027a0 msgr2=0x7f11e0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e01027a0 0x7f11e0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11de59c640 1 -- 172.21.15.31:0/4160443454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e019d3a0 con 0x7f11e0101df0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.349+0000 7f11ded9d640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 0x7f11e019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11de59c640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11d000e9f0 tx=0x7f11d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000cd80 con 0x7f11e0101df0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e019d690 con 0x7f11e0101df0 2024-09-15T23:59:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d00108e0 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0018920 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.350+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a4f70 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.351+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e01021d0 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.352+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f11d0010a90 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.353+0000 7f11cf7fe640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 0x7f11c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.353+0000 7f11ddd9b640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 0x7f11c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.353+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f11d00cfd90 con 0x7f11e0101df0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.353+0000 7f11ddd9b640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 0x7f11c80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11e00fdda0 tx=0x7f11d400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.357+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11d00541b0 con 0x7f11e0101df0 2024-09-15T23:59:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.360+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f2db802d7c0 con 0x7f2db8102790 2024-09-15T23:59:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.361+0000 7f2db4ff9640 1 -- 172.21.15.31:0/2521816613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2dac09bbf0 con 0x7f2db8102790 2024-09-15T23:59:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:266287972388 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 msgr2=0x7f2da00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 0x7f2da00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2da800bbe0 tx=0x7f2da800b390 comp rx=0 tx=0).stop 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 msgr2=0x7f2db81988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c890 tx=0x7f2dac00cd50 comp rx=0 tx=0).stop 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 shutdown_connections 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2da00a32d0 0x7f2da00a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db8106f10 0x7f2db819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2db8102790 0x7f2db81988a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 --2- 172.21.15.31:0/2521816613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db8101de0 0x7f2db8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 >> 172.21.15.31:0/2521816613 conn(0x7f2db80fd8c0 msgr2=0x7f2db80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 shutdown_connections 2024-09-15T23:59:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.363+0000 7f2dbde12640 1 -- 172.21.15.31:0/2521816613 wait complete. 2024-09-15T23:59:10.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.536+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f11e0199750 con 0x7f11e0101df0 2024-09-15T23:59:10.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.537+0000 7f11cf7fe640 1 -- 172.21.15.31:0/4160443454 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f11d009ca00 con 0x7f11e0101df0 2024-09-15T23:59:10.538 INFO:teuthology.orchestra.run.smithi031.stdout:103079215171 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 msgr2=0x7f11c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 0x7f11c80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11e00fdda0 tx=0x7f11d400a250 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 msgr2=0x7f11e01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f11d000e9f0 tx=0x7f11d000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 shutdown_connections 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f11c80a32d0 0x7f11c80a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11e0106f20 0x7f11e019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e01027a0 0x7f11e0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 --2- 172.21.15.31:0/4160443454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0101df0 0x7f11e01983f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 >> 172.21.15.31:0/4160443454 conn(0x7f11e00fd8f0 msgr2=0x7f11e00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 shutdown_connections 2024-09-15T23:59:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:10.539+0000 7f11e4816640 1 -- 172.21.15.31:0/4160443454 wait complete. 2024-09-15T23:59:11.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:11 smithi031 ceph-mon[31203]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:11.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:11 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2521816613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T23:59:11.190 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:11 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4160443454' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 -- 172.21.15.31:0/1559622864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1036e0 msgr2=0x7f55ec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 --2- 172.21.15.31:0/1559622864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1036e0 0x7f55ec107b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009930 tx=0x7f55e002a420 comp rx=0 tx=0).stop 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 -- 172.21.15.31:0/1559622864 shutdown_connections 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 --2- 172.21.15.31:0/1559622864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1082a0 0x7f55ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 --2- 172.21.15.31:0/1559622864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1036e0 0x7f55ec107b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 --2- 172.21.15.31:0/1559622864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 -- 172.21.15.31:0/1559622864 >> 172.21.15.31:0/1559622864 conn(0x7f55ec0fc7d0 msgr2=0x7f55ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 -- 172.21.15.31:0/1559622864 shutdown_connections 2024-09-15T23:59:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.203+0000 7f55f47db640 1 -- 172.21.15.31:0/1559622864 wait complete. 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 Processor -- start 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 -- start start 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1036e0 0x7f55ec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec111f10 con 0x7f55ec1082a0 2024-09-15T23:59:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec111d90 con 0x7f55ec1036e0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f47db640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec112090 con 0x7f55ec102d30 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f2d78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f2d78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58398/0 (socket says 172.21.15.31:58398) 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f2d78640 1 -- 172.21.15.31:0/896774280 learned_addr learned my addr 172.21.15.31:0/896774280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f2577640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.204+0000 7f55f1d76640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1036e0 0x7f55ec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 -- 172.21.15.31:0/896774280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 msgr2=0x7f55ec198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 -- 172.21.15.31:0/896774280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1036e0 msgr2=0x7f55ec198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1036e0 0x7f55ec198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 -- 172.21.15.31:0/896774280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec19d4e0 con 0x7f55ec1082a0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2577640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f2d78640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55e400d750 tx=0x7f55e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4017070 con 0x7f55ec1082a0 2024-09-15T23:59:11.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55e40054d0 con 0x7f55ec1082a0 2024-09-15T23:59:11.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec19d7d0 con 0x7f55ec1082a0 2024-09-15T23:59:11.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004c60 con 0x7f55ec1082a0 2024-09-15T23:59:11.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.205+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec1a50b0 con 0x7f55ec1082a0 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.206+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec06a850 con 0x7f55ec1082a0 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.207+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f55e4004e00 con 0x7f55ec1082a0 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.208+0000 7f55eb7fe640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 0x7f55d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.208+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f55e40cf230 con 0x7f55ec1082a0 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.208+0000 7f55f2577640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 0x7f55d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.209+0000 7f55f2577640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 0x7f55d40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009fd0 tx=0x7f55dc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.212+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55e409bfa0 con 0x7f55ec1082a0 2024-09-15T23:59:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:11 smithi081 ceph-mon[35060]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:11 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2521816613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T23:59:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:11 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4160443454' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- 172.21.15.31:0/290379252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c102df0 msgr2=0x7f235c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 --2- 172.21.15.31:0/290379252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c102df0 0x7f235c1031d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2358009900 tx=0x7f235802a410 comp rx=0 tx=0).stop 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- 172.21.15.31:0/290379252 shutdown_connections 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 --2- 172.21.15.31:0/290379252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 0x7f235c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 --2- 172.21.15.31:0/290379252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c1037a0 0x7f235c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 --2- 172.21.15.31:0/290379252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c102df0 0x7f235c1031d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- 172.21.15.31:0/290379252 >> 172.21.15.31:0/290379252 conn(0x7f235c0fc7d0 msgr2=0x7f235c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- 172.21.15.31:0/290379252 shutdown_connections 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- 172.21.15.31:0/290379252 wait complete. 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 Processor -- start 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.104+0000 7f23638b6640 1 -- start start 2024-09-15T23:59:11.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 0x7f235c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 0x7f235c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f235c112000 con 0x7f235c1037a0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f235c111e80 con 0x7f235c102df0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f235c112180 con 0x7f235c10c2f0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58380/0 (socket says 172.21.15.31:58380) 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 -- 172.21.15.31:0/4149223852 learned_addr learned my addr 172.21.15.31:0/4149223852 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2361e53640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 0x7f235c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2361652640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 0x7f235c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 -- 172.21.15.31:0/4149223852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 msgr2=0x7f235c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 0x7f235c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 -- 172.21.15.31:0/4149223852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 msgr2=0x7f235c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 0x7f235c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 -- 172.21.15.31:0/4149223852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c19d530 con 0x7f235c1037a0 2024-09-15T23:59:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2361652640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 0x7f235c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f2360e51640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f234c006fd0 tx=0x7f234c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c011830 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f234c016070 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c19d820 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c010df0 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.105+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1a5100 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.106+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c06a790 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.108+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f234c0119d0 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.108+0000 7f23527fc640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 0x7f23480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.108+0000 7f2361652640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 0x7f23480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.108+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f234c0cf550 con 0x7f235c1037a0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.109+0000 7f2361652640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 0x7f23480a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2358006010 tx=0x7f2358031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.111+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f234c09c3d0 con 0x7f235c1037a0 2024-09-15T23:59:11.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.317+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f235c1031d0 con 0x7f235c1037a0 2024-09-15T23:59:11.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.317+0000 7f23527fc640 1 -- 172.21.15.31:0/4149223852 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f235c1031d0 con 0x7f235c1037a0 2024-09-15T23:59:11.318 INFO:teuthology.orchestra.run.smithi031.stdout:34359738450 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 msgr2=0x7f23480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 0x7f23480a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2358006010 tx=0x7f2358031400 comp rx=0 tx=0).stop 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 msgr2=0x7f235c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f234c006fd0 tx=0x7f234c00b8e0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 shutdown_connections 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f23480a32d0 0x7f23480a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f235c10c2f0 0x7f235c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c1037a0 0x7f235c198ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 --2- 172.21.15.31:0/4149223852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235c102df0 0x7f235c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 >> 172.21.15.31:0/4149223852 conn(0x7f235c0fc7d0 msgr2=0x7f235c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:11.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 shutdown_connections 2024-09-15T23:59:11.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.319+0000 7f23638b6640 1 -- 172.21.15.31:0/4149223852 wait complete. 2024-09-15T23:59:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:11 smithi102 ceph-mon[34458]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:11 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2521816613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T23:59:11.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:11 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4160443454' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T23:59:11.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.412+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f55ec1998c0 con 0x7f55ec1082a0 2024-09-15T23:59:11.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.413+0000 7f55eb7fe640 1 -- 172.21.15.31:0/896774280 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55ec1998c0 con 0x7f55ec1082a0 2024-09-15T23:59:11.414 INFO:teuthology.orchestra.run.smithi031.stdout:236223201321 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 msgr2=0x7f55d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 0x7f55d40a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009fd0 tx=0x7f55dc009680 comp rx=0 tx=0).stop 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 msgr2=0x7f55ec19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55e400d750 tx=0x7f55e400dc10 comp rx=0 tx=0).stop 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 shutdown_connections 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f55d40a32d0 0x7f55d40a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55ec1082a0 0x7f55ec19ce00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ec1036e0 0x7f55ec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 --2- 172.21.15.31:0/896774280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55ec102d30 0x7f55ec198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:11.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 >> 172.21.15.31:0/896774280 conn(0x7f55ec0fc7d0 msgr2=0x7f55ec103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:11.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 shutdown_connections 2024-09-15T23:59:11.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:11.415+0000 7f55f47db640 1 -- 172.21.15.31:0/896774280 wait complete. 2024-09-15T23:59:11.878 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:11.878 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:12.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:12 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4149223852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T23:59:12.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:12 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/896774280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T23:59:12.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201318 got 236223201321 for osd.8 2024-09-15T23:59:12.341 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:12.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:12 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4149223852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T23:59:12.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:12 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/896774280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T23:59:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:12 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4149223852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T23:59:12.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:12 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/896774280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T23:59:12.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738450 for osd.0 2024-09-15T23:59:12.774 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:13 smithi031 ceph-mon[31203]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:13.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:13 smithi102 ceph-mon[34458]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:13 smithi081 ceph-mon[35060]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 -- 172.21.15.31:0/1071354210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701082a0 msgr2=0x7fd97010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 --2- 172.21.15.31:0/1071354210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701082a0 0x7fd97010ec40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd968009cd0 tx=0x7fd96802a8e0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 -- 172.21.15.31:0/1071354210 shutdown_connections 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 --2- 172.21.15.31:0/1071354210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701082a0 0x7fd97010ec40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 --2- 172.21.15.31:0/1071354210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9701036e0 0x7fd970107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 --2- 172.21.15.31:0/1071354210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd970102d30 0x7fd970103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 -- 172.21.15.31:0/1071354210 >> 172.21.15.31:0/1071354210 conn(0x7fd9700fc7d0 msgr2=0x7fd9700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 -- 172.21.15.31:0/1071354210 shutdown_connections 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 -- 172.21.15.31:0/1071354210 wait complete. 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.566+0000 7fd975190640 1 Processor -- start 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 -- start start 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 0x7fd970100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 0x7fd970101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd970111f50 con 0x7fd9701036e0 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd970111dd0 con 0x7fd970102d30 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd975190640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9701120d0 con 0x7fd9701082a0 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34862/0 (socket says 172.21.15.31:34862) 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 -- 172.21.15.31:0/3217839664 learned_addr learned my addr 172.21.15.31:0/3217839664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96ed9d640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 0x7fd970100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96f59e640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 0x7fd970101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 -- 172.21.15.31:0/3217839664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 msgr2=0x7fd970101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 0x7fd970101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 -- 172.21.15.31:0/3217839664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 msgr2=0x7fd970100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 0x7fd970100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.567+0000 7fd96e59c640 1 -- 172.21.15.31:0/3217839664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9701a6950 con 0x7fd9701036e0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd96f59e640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 0x7fd970101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd96ed9d640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 0x7fd970100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd96e59c640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400ccc0 con 0x7fd9701036e0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9701a6c40 con 0x7fd9701036e0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9640108e0 con 0x7fd9701036e0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640187c0 con 0x7fd9701036e0 2024-09-15T23:59:13.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.568+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701a7180 con 0x7fd9701036e0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.569+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd970103110 con 0x7fd9701036e0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.571+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd964010a90 con 0x7fd9701036e0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.571+0000 7fd957fff640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 0x7fd9580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.571+0000 7fd96ed9d640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 0x7fd9580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.571+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd9640d0140 con 0x7fd9701036e0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.572+0000 7fd96ed9d640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 0x7fd9580a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9600058c0 tx=0x7fd960005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:13.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.578+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd96409ceb0 con 0x7fd9701036e0 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 -- 172.21.15.31:0/2353027557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481082a0 msgr2=0x7f6d4810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2353027557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481082a0 0x7f6d4810ec40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009cd0 tx=0x7f6d4002a8e0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 -- 172.21.15.31:0/2353027557 shutdown_connections 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2353027557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481082a0 0x7f6d4810ec40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2353027557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481036e0 0x7f6d48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2353027557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 0x7f6d48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 -- 172.21.15.31:0/2353027557 >> 172.21.15.31:0/2353027557 conn(0x7f6d480fc7d0 msgr2=0x7f6d480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 -- 172.21.15.31:0/2353027557 shutdown_connections 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.462+0000 7f6d4e207640 1 -- 172.21.15.31:0/2353027557 wait complete. 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 Processor -- start 2024-09-15T23:59:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 -- start start 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 0x7f6d481984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 0x7f6d4819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48111f50 con 0x7f6d481036e0 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48111dd0 con 0x7f6d48102d30 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d4e207640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481120d0 con 0x7f6d481082a0 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.463+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34846/0 (socket says 172.21.15.31:34846) 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 -- 172.21.15.31:0/2938240566 learned_addr learned my addr 172.21.15.31:0/2938240566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d477fe640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 0x7f6d481984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d47fff640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 0x7f6d4819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 -- 172.21.15.31:0/2938240566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 msgr2=0x7f6d4819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 0x7f6d4819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 -- 172.21.15.31:0/2938240566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 msgr2=0x7f6d481984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 0x7f6d481984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 -- 172.21.15.31:0/2938240566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4819d460 con 0x7f6d481036e0 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d47fff640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 0x7f6d4819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d46ffd640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c970 tx=0x7f6d3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004220 con 0x7f6d481036e0 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c0043c0 con 0x7f6d481036e0 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stdout:317827579930 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4819d750 con 0x7f6d481036e0 2024-09-15T23:59:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004a00 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.464+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a5030 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.465+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d48103110 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.466+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6d3c0079b0 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.467+0000 7f6d44ff9640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 0x7f6d300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.467+0000 7f6d477fe640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 0x7f6d300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.468+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6d3c0cf590 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.468+0000 7f6d477fe640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 0x7f6d300a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bbe0 tx=0x7f6d3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.471+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d3c014030 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.654+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6d4810a2b0 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.655+0000 7f6d44ff9640 1 -- 172.21.15.31:0/2938240566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6d3c09c300 con 0x7f6d481036e0 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 msgr2=0x7f6d300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 0x7f6d300a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800bbe0 tx=0x7f6d3800b390 comp rx=0 tx=0).stop 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 msgr2=0x7f6d481989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c970 tx=0x7f6d3c00ce30 comp rx=0 tx=0).stop 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 shutdown_connections 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f6d300a32d0 0x7f6d300a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d481082a0 0x7f6d4819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d481036e0 0x7f6d481989f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 --2- 172.21.15.31:0/2938240566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d48102d30 0x7f6d481984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.657+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 >> 172.21.15.31:0/2938240566 conn(0x7f6d480fc7d0 msgr2=0x7f6d48105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.658+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 shutdown_connections 2024-09-15T23:59:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.658+0000 7f6d4e207640 1 -- 172.21.15.31:0/2938240566 wait complete. 2024-09-15T23:59:13.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.768+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd97010a2b0 con 0x7fd9701036e0 2024-09-15T23:59:13.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.768+0000 7fd957fff640 1 -- 172.21.15.31:0/3217839664 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd96409c7b0 con 0x7fd9701036e0 2024-09-15T23:59:13.769 INFO:teuthology.orchestra.run.smithi031.stdout:154618822712 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 msgr2=0x7fd9580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 0x7fd9580a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9600058c0 tx=0x7fd960005850 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 msgr2=0x7fd970100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 shutdown_connections 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fd9580a32d0 0x7fd9580a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9701082a0 0x7fd970101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9701036e0 0x7fd970100ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 --2- 172.21.15.31:0/3217839664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd970102d30 0x7fd970100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:13.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 >> 172.21.15.31:0/3217839664 conn(0x7fd9700fc7d0 msgr2=0x7fd970105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:13.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 shutdown_connections 2024-09-15T23:59:13.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:13.770+0000 7fd975190640 1 -- 172.21.15.31:0/3217839664 wait complete. 2024-09-15T23:59:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2938240566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T23:59:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3217839664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T23:59:14.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2938240566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T23:59:14.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3217839664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T23:59:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2938240566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T23:59:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3217839664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T23:59:14.746 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972386 got 266287972388 for osd.9 2024-09-15T23:59:14.746 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:15.151 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:15 smithi031 ceph-mon[31203]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-15T23:59:15.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215171 for osd.3 2024-09-15T23:59:15.183 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:15 smithi102 ceph-mon[34458]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-15T23:59:15.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:15 smithi081 ceph-mon[35060]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-15T23:59:15.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822710 got 154618822712 for osd.5 2024-09-15T23:59:15.644 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 -- 172.21.15.31:0/3409097481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 msgr2=0x7f2afc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 --2- 172.21.15.31:0/3409097481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc107b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009930 tx=0x7f2af002a420 comp rx=0 tx=0).stop 2024-09-15T23:59:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 -- 172.21.15.31:0/3409097481 shutdown_connections 2024-09-15T23:59:15.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 --2- 172.21.15.31:0/3409097481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc1082a0 0x7f2afc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 --2- 172.21.15.31:0/3409097481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc107b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 --2- 172.21.15.31:0/3409097481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc102d30 0x7f2afc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 -- 172.21.15.31:0/3409097481 >> 172.21.15.31:0/3409097481 conn(0x7f2afc0fc7d0 msgr2=0x7f2afc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 -- 172.21.15.31:0/3409097481 shutdown_connections 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.842+0000 7f2b03027640 1 -- 172.21.15.31:0/3409097481 wait complete. 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 Processor -- start 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 -- start start 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc102d30 0x7f2afc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc1082a0 0x7f2afc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc111f50 con 0x7f2afc1036e0 2024-09-15T23:59:15.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc111dd0 con 0x7f2afc102d30 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b03027640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1120d0 con 0x7f2afc1082a0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34886/0 (socket says 172.21.15.31:34886) 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 -- 172.21.15.31:0/2177699875 learned_addr learned my addr 172.21.15.31:0/2177699875 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b00dc3640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc102d30 0x7f2afc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2b015c4640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc1082a0 0x7f2afc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 -- 172.21.15.31:0/2177699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc1082a0 msgr2=0x7f2afc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc1082a0 0x7f2afc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 -- 172.21.15.31:0/2177699875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc102d30 msgr2=0x7f2afc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc102d30 0x7f2afc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.843+0000 7f2afbfff640 1 -- 172.21.15.31:0/2177699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1a69d0 con 0x7f2afc1036e0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2afbfff640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2af0031440 tx=0x7f2af0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:15.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af003f030 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2af0003aa0 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0035070 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a6c60 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.844+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a70f0 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.845+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc103110 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.847+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2af0042030 con 0x7f2afc1036e0 2024-09-15T23:59:15.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.847+0000 7f2af9ffb640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 0x7f2ae40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.848+0000 7f2b00dc3640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 0x7f2ae40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.848+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2af00effe0 con 0x7f2afc1036e0 2024-09-15T23:59:15.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.848+0000 7f2b00dc3640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 0x7f2ae40a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005fd0 tx=0x7f2aec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:15.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:15.852+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2af00bcd50 con 0x7f2afc1036e0 2024-09-15T23:59:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.034+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2afc10a2b0 con 0x7f2afc1036e0 2024-09-15T23:59:16.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.034+0000 7f2af9ffb640 1 -- 172.21.15.31:0/2177699875 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2af00bc650 con 0x7f2afc1036e0 2024-09-15T23:59:16.108 INFO:teuthology.orchestra.run.smithi031.stdout:133143986237 2024-09-15T23:59:16.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 msgr2=0x7f2ae40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:16.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 0x7f2ae40a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005fd0 tx=0x7f2aec00b390 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 msgr2=0x7f2afc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2af0031440 tx=0x7f2af0004270 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 shutdown_connections 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2ae40a3350 0x7f2ae40a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2afc1082a0 0x7f2afc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc1036e0 0x7f2afc100ed0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 --2- 172.21.15.31:0/2177699875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2afc102d30 0x7f2afc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 >> 172.21.15.31:0/2177699875 conn(0x7f2afc0fc7d0 msgr2=0x7f2afc105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.036+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 shutdown_connections 2024-09-15T23:59:16.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:16.037+0000 7f2b03027640 1 -- 172.21.15.31:0/2177699875 wait complete. 2024-09-15T23:59:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:16 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2177699875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T23:59:16.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:16 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2177699875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T23:59:16.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579928 got 317827579930 for osd.11 2024-09-15T23:59:16.533 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:16.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:16 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2177699875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T23:59:17.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986237 for osd.4 2024-09-15T23:59:17.103 DEBUG:teuthology.parallel:result is None 2024-09-15T23:59:17.103 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T23:59:17.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T23:59:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:17 smithi102 ceph-mon[34458]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:17.442 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:17 smithi031 ceph-mon[31203]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:17 smithi081 ceph-mon[35060]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T23:59:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- 172.21.15.31:0/4001071460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 msgr2=0x7fcd1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/4001071460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd1810ea90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009ca0 tx=0x7fcd1002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- 172.21.15.31:0/4001071460 shutdown_connections 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/4001071460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd1810ea90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/4001071460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 0x7fcd181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/4001071460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 0x7fcd181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- 172.21.15.31:0/4001071460 >> 172.21.15.31:0/4001071460 conn(0x7fcd180fd8c0 msgr2=0x7fcd180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- 172.21.15.31:0/4001071460 shutdown_connections 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- 172.21.15.31:0/4001071460 wait complete. 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 Processor -- start 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.271+0000 7fcd1e7df640 1 -- start start 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 0x7fcd180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 0x7fcd18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111e20 con 0x7fcd18106f10 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111ca0 con 0x7fcd18102790 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1e7df640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111fa0 con 0x7fcd18101de0 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34898/0 (socket says 172.21.15.31:34898) 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 -- 172.21.15.31:0/1221847002 learned_addr learned my addr 172.21.15.31:0/1221847002 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd17fff640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 0x7fcd180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd177fe640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 0x7fcd18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 -- 172.21.15.31:0/1221847002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 msgr2=0x7fcd180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 0x7fcd180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 -- 172.21.15.31:0/1221847002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 msgr2=0x7fcd18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 0x7fcd18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.272+0000 7fcd1cd7c640 1 -- 172.21.15.31:0/1221847002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd181a66c0 con 0x7fcd18106f10 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd1cd7c640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009dc0 tx=0x7fcd10003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004040 con 0x7fcd18106f10 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd10004690 con 0x7fcd18106f10 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd181a6950 con 0x7fcd18106f10 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10030c90 con 0x7fcd18106f10 2024-09-15T23:59:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.273+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd181a6e90 con 0x7fcd18106f10 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.274+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd181021c0 con 0x7fcd18106f10 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.275+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcd100041e0 con 0x7fcd18106f10 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.276+0000 7fcd157fa640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 0x7fcd000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.276+0000 7fcd17fff640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 0x7fcd000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.276+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd100f0fc0 con 0x7fcd18106f10 2024-09-15T23:59:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.276+0000 7fcd17fff640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 0x7fcd000a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800abe0 tx=0x7fcd0800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.280+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd100bde40 con 0x7fcd18106f10 2024-09-15T23:59:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.494+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd180febd0 con 0x7fcd000a32d0 2024-09-15T23:59:18.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.502+0000 7fcd157fa640 1 -- 172.21.15.31:0/1221847002 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258941 (secure 0 0 0) 0x7fcd180febd0 con 0x7fcd000a32d0 2024-09-15T23:59:18.505 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:18.506 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-15T23:59:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 msgr2=0x7fcd000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 0x7fcd000a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800abe0 tx=0x7fcd0800a2d0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 msgr2=0x7fcd18074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:18.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009dc0 tx=0x7fcd10003ba0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 shutdown_connections 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd000a32d0 0x7fcd000a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd18106f10 0x7fcd18074f70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd18102790 0x7fcd18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 --2- 172.21.15.31:0/1221847002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd18101de0 0x7fcd180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 >> 172.21.15.31:0/1221847002 conn(0x7fcd180fd8c0 msgr2=0x7fcd180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 shutdown_connections 2024-09-15T23:59:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:18.505+0000 7fcd1e7df640 1 -- 172.21.15.31:0/1221847002 wait complete. 2024-09-15T23:59:19.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-15T23:59:17.215365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659169,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":678,"num_read_kb":463,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347556,"kb_used_data":10788,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594268,"statfs":{"total":1151940427776,"available":1151584530432,"internally_reserved":0,"allocated":11046912,"data_stored":6971919,"data_compressed":112518,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":344634682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015765"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154743+0000","last_change":"2024-09-15T23:58:11.160319+0000","last_active":"2024-09-15T23:58:12.154743+0000","last_peered":"2024-09-15T23:58:12.154743+0000","last_clean":"2024-09-15T23:58:12.154743+0000","last_became_active":"2024-09-15T23:58:11.159572+0000","last_became_peered":"2024-09-15T23:58:11.159572+0000","last_unstale":"2024-09-15T23:58:12.154743+0000","last_undegraded":"2024-09-15T23:58:12.154743+0000","last_fullsized":"2024-09-15T23:58:12.154743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:50.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150768+0000","last_change":"2024-09-15T23:58:11.152044+0000","last_active":"2024-09-15T23:58:12.150768+0000","last_peered":"2024-09-15T23:58:12.150768+0000","last_clean":"2024-09-15T23:58:12.150768+0000","last_became_active":"2024-09-15T23:58:11.151472+0000","last_became_peered":"2024-09-15T23:58:11.151472+0000","last_unstale":"2024-09-15T23:58:12.150768+0000","last_undegraded":"2024-09-15T23:58:12.150768+0000","last_fullsized":"2024-09-15T23:58:12.150768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:51.502165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150571+0000","last_change":"2024-09-15T23:58:11.151854+0000","last_active":"2024-09-15T23:58:12.150571+0000","last_peered":"2024-09-15T23:58:12.150571+0000","last_clean":"2024-09-15T23:58:12.150571+0000","last_became_active":"2024-09-15T23:58:11.151418+0000","last_became_peered":"2024-09-15T23:58:11.151418+0000","last_unstale":"2024-09-15T23:58:12.150571+0000","last_undegraded":"2024-09-15T23:58:12.150571+0000","last_fullsized":"2024-09-15T23:58:12.150571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:48.513653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149324+0000","last_change":"2024-09-15T23:58:11.162524+0000","last_active":"2024-09-15T23:58:12.149324+0000","last_peered":"2024-09-15T23:58:12.149324+0000","last_clean":"2024-09-15T23:58:12.149324+0000","last_became_active":"2024-09-15T23:58:11.161994+0000","last_became_peered":"2024-09-15T23:58:11.161994+0000","last_unstale":"2024-09-15T23:58:12.149324+0000","last_undegraded":"2024-09-15T23:58:12.149324+0000","last_fullsized":"2024-09-15T23:58:12.149324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:25.333515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150250+0000","last_change":"2024-09-15T23:58:11.143229+0000","last_active":"2024-09-15T23:58:12.150250+0000","last_peered":"2024-09-15T23:58:12.150250+0000","last_clean":"2024-09-15T23:58:12.150250+0000","last_became_active":"2024-09-15T23:58:11.142346+0000","last_became_peered":"2024-09-15T23:58:11.142346+0000","last_unstale":"2024-09-15T23:58:12.150250+0000","last_undegraded":"2024-09-15T23:58:12.150250+0000","last_fullsized":"2024-09-15T23:58:12.150250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:51.414916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136944+0000","last_change":"2024-09-15T23:58:11.152092+0000","last_active":"2024-09-15T23:58:12.136944+0000","last_peered":"2024-09-15T23:58:12.136944+0000","last_clean":"2024-09-15T23:58:12.136944+0000","last_became_active":"2024-09-15T23:58:11.151769+0000","last_became_peered":"2024-09-15T23:58:11.151769+0000","last_unstale":"2024-09-15T23:58:12.136944+0000","last_undegraded":"2024-09-15T23:58:12.136944+0000","last_fullsized":"2024-09-15T23:58:12.136944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:42.690545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150191+0000","last_change":"2024-09-15T23:58:11.158671+0000","last_active":"2024-09-15T23:58:12.150191+0000","last_peered":"2024-09-15T23:58:12.150191+0000","last_clean":"2024-09-15T23:58:12.150191+0000","last_became_active":"2024-09-15T23:58:11.158347+0000","last_became_peered":"2024-09-15T23:58:11.158347+0000","last_unstale":"2024-09-15T23:58:12.150191+0000","last_undegraded":"2024-09-15T23:58:12.150191+0000","last_fullsized":"2024-09-15T23:58:12.150191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:52.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136298+0000","last_change":"2024-09-15T23:58:11.149725+0000","last_active":"2024-09-15T23:58:12.136298+0000","last_peered":"2024-09-15T23:58:12.136298+0000","last_clean":"2024-09-15T23:58:12.136298+0000","last_became_active":"2024-09-15T23:58:11.149388+0000","last_became_peered":"2024-09-15T23:58:11.149388+0000","last_unstale":"2024-09-15T23:58:12.136298+0000","last_undegraded":"2024-09-15T23:58:12.136298+0000","last_fullsized":"2024-09-15T23:58:12.136298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:54.807443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149128+0000","last_change":"2024-09-15T23:58:11.155976+0000","last_active":"2024-09-15T23:58:12.149128+0000","last_peered":"2024-09-15T23:58:12.149128+0000","last_clean":"2024-09-15T23:58:12.149128+0000","last_became_active":"2024-09-15T23:58:11.155659+0000","last_became_peered":"2024-09-15T23:58:11.155659+0000","last_unstale":"2024-09-15T23:58:12.149128+0000","last_undegraded":"2024-09-15T23:58:12.149128+0000","last_fullsized":"2024-09-15T23:58:12.149128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:21.524908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137172+0000","last_change":"2024-09-15T23:58:11.157326+0000","last_active":"2024-09-15T23:58:12.137172+0000","last_peered":"2024-09-15T23:58:12.137172+0000","last_clean":"2024-09-15T23:58:12.137172+0000","last_became_active":"2024-09-15T23:58:11.156944+0000","last_became_peered":"2024-09-15T23:58:11.156944+0000","last_unstale":"2024-09-15T23:58:12.137172+0000","last_undegraded":"2024-09-15T23:58:12.137172+0000","last_fullsized":"2024-09-15T23:58:12.137172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:49.913437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151176+0000","last_change":"2024-09-15T23:58:11.154418+0000","last_active":"2024-09-15T23:58:12.151176+0000","last_peered":"2024-09-15T23:58:12.151176+0000","last_clean":"2024-09-15T23:58:12.151176+0000","last_became_active":"2024-09-15T23:58:11.154081+0000","last_became_peered":"2024-09-15T23:58:11.154081+0000","last_unstale":"2024-09-15T23:58:12.151176+0000","last_undegraded":"2024-09-15T23:58:12.151176+0000","last_fullsized":"2024-09-15T23:58:12.151176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:09.920736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149482+0000","last_change":"2024-09-15T23:58:11.158224+0000","last_active":"2024-09-15T23:58:12.149482+0000","last_peered":"2024-09-15T23:58:12.149482+0000","last_clean":"2024-09-15T23:58:12.149482+0000","last_became_active":"2024-09-15T23:58:11.156076+0000","last_became_peered":"2024-09-15T23:58:11.156076+0000","last_unstale":"2024-09-15T23:58:12.149482+0000","last_undegraded":"2024-09-15T23:58:12.149482+0000","last_fullsized":"2024-09-15T23:58:12.149482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:29.448669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148503+0000","last_change":"2024-09-15T23:58:11.151904+0000","last_active":"2024-09-15T23:58:12.148503+0000","last_peered":"2024-09-15T23:58:12.148503+0000","last_clean":"2024-09-15T23:58:12.148503+0000","last_became_active":"2024-09-15T23:58:11.151564+0000","last_became_peered":"2024-09-15T23:58:11.151564+0000","last_unstale":"2024-09-15T23:58:12.148503+0000","last_undegraded":"2024-09-15T23:58:12.148503+0000","last_fullsized":"2024-09-15T23:58:12.148503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:11.297350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151021+0000","last_change":"2024-09-15T23:58:11.148139+0000","last_active":"2024-09-15T23:58:12.151021+0000","last_peered":"2024-09-15T23:58:12.151021+0000","last_clean":"2024-09-15T23:58:12.151021+0000","last_became_active":"2024-09-15T23:58:11.147820+0000","last_became_peered":"2024-09-15T23:58:11.147820+0000","last_unstale":"2024-09-15T23:58:12.151021+0000","last_undegraded":"2024-09-15T23:58:12.151021+0000","last_fullsized":"2024-09-15T23:58:12.151021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:33.786315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150504+0000","last_change":"2024-09-15T23:58:11.160284+0000","last_active":"2024-09-15T23:58:12.150504+0000","last_peered":"2024-09-15T23:58:12.150504+0000","last_clean":"2024-09-15T23:58:12.150504+0000","last_became_active":"2024-09-15T23:58:11.159931+0000","last_became_peered":"2024-09-15T23:58:11.159931+0000","last_unstale":"2024-09-15T23:58:12.150504+0000","last_undegraded":"2024-09-15T23:58:12.150504+0000","last_fullsized":"2024-09-15T23:58:12.150504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:35.381030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147613+0000","last_change":"2024-09-15T23:58:11.162637+0000","last_active":"2024-09-15T23:58:12.147613+0000","last_peered":"2024-09-15T23:58:12.147613+0000","last_clean":"2024-09-15T23:58:12.147613+0000","last_became_active":"2024-09-15T23:58:11.162322+0000","last_became_peered":"2024-09-15T23:58:11.162322+0000","last_unstale":"2024-09-15T23:58:12.147613+0000","last_undegraded":"2024-09-15T23:58:12.147613+0000","last_fullsized":"2024-09-15T23:58:12.147613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:50.489012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154813+0000","last_change":"2024-09-15T23:58:11.160068+0000","last_active":"2024-09-15T23:58:12.154813+0000","last_peered":"2024-09-15T23:58:12.154813+0000","last_clean":"2024-09-15T23:58:12.154813+0000","last_became_active":"2024-09-15T23:58:11.159553+0000","last_became_peered":"2024-09-15T23:58:11.159553+0000","last_unstale":"2024-09-15T23:58:12.154813+0000","last_undegraded":"2024-09-15T23:58:12.154813+0000","last_fullsized":"2024-09-15T23:58:12.154813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:31.962049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147516+0000","last_change":"2024-09-15T23:58:11.148755+0000","last_active":"2024-09-15T23:58:12.147516+0000","last_peered":"2024-09-15T23:58:12.147516+0000","last_clean":"2024-09-15T23:58:12.147516+0000","last_became_active":"2024-09-15T23:58:11.148428+0000","last_became_peered":"2024-09-15T23:58:11.148428+0000","last_unstale":"2024-09-15T23:58:12.147516+0000","last_undegraded":"2024-09-15T23:58:12.147516+0000","last_fullsized":"2024-09-15T23:58:12.147516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.119847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148320+0000","last_change":"2024-09-15T23:58:11.143374+0000","last_active":"2024-09-15T23:58:12.148320+0000","last_peered":"2024-09-15T23:58:12.148320+0000","last_clean":"2024-09-15T23:58:12.148320+0000","last_became_active":"2024-09-15T23:58:11.143032+0000","last_became_peered":"2024-09-15T23:58:11.143032+0000","last_unstale":"2024-09-15T23:58:12.148320+0000","last_undegraded":"2024-09-15T23:58:12.148320+0000","last_fullsized":"2024-09-15T23:58:12.148320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:29.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153498+0000","last_change":"2024-09-15T23:58:11.160524+0000","last_active":"2024-09-15T23:58:12.153498+0000","last_peered":"2024-09-15T23:58:12.153498+0000","last_clean":"2024-09-15T23:58:12.153498+0000","last_became_active":"2024-09-15T23:58:11.160141+0000","last_became_peered":"2024-09-15T23:58:11.160141+0000","last_unstale":"2024-09-15T23:58:12.153498+0000","last_undegraded":"2024-09-15T23:58:12.153498+0000","last_fullsized":"2024-09-15T23:58:12.153498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:36.730764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148654+0000","last_change":"2024-09-15T23:58:11.153493+0000","last_active":"2024-09-15T23:58:12.148654+0000","last_peered":"2024-09-15T23:58:12.148654+0000","last_clean":"2024-09-15T23:58:12.148654+0000","last_became_active":"2024-09-15T23:58:11.153026+0000","last_became_peered":"2024-09-15T23:58:11.153026+0000","last_unstale":"2024-09-15T23:58:12.148654+0000","last_undegraded":"2024-09-15T23:58:12.148654+0000","last_fullsized":"2024-09-15T23:58:12.148654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:15.700186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148212+0000","last_change":"2024-09-15T23:58:11.159991+0000","last_active":"2024-09-15T23:58:12.148212+0000","last_peered":"2024-09-15T23:58:12.148212+0000","last_clean":"2024-09-15T23:58:12.148212+0000","last_became_active":"2024-09-15T23:58:11.159656+0000","last_became_peered":"2024-09-15T23:58:11.159656+0000","last_unstale":"2024-09-15T23:58:12.148212+0000","last_undegraded":"2024-09-15T23:58:12.148212+0000","last_fullsized":"2024-09-15T23:58:12.148212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:53.065297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136354+0000","last_change":"2024-09-15T23:58:11.154058+0000","last_active":"2024-09-15T23:58:12.136354+0000","last_peered":"2024-09-15T23:58:12.136354+0000","last_clean":"2024-09-15T23:58:12.136354+0000","last_became_active":"2024-09-15T23:58:11.153769+0000","last_became_peered":"2024-09-15T23:58:11.153769+0000","last_unstale":"2024-09-15T23:58:12.136354+0000","last_undegraded":"2024-09-15T23:58:12.136354+0000","last_fullsized":"2024-09-15T23:58:12.136354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:03:54.768080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136887+0000","last_change":"2024-09-15T23:58:11.148173+0000","last_active":"2024-09-15T23:58:12.136887+0000","last_peered":"2024-09-15T23:58:12.136887+0000","last_clean":"2024-09-15T23:58:12.136887+0000","last_became_active":"2024-09-15T23:58:11.147843+0000","last_became_peered":"2024-09-15T23:58:11.147843+0000","last_unstale":"2024-09-15T23:58:12.136887+0000","last_undegraded":"2024-09-15T23:58:12.136887+0000","last_fullsized":"2024-09-15T23:58:12.136887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:29.789888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136563+0000","last_change":"2024-09-15T23:58:11.162113+0000","last_active":"2024-09-15T23:58:12.136563+0000","last_peered":"2024-09-15T23:58:12.136563+0000","last_clean":"2024-09-15T23:58:12.136563+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.136563+0000","last_undegraded":"2024-09-15T23:58:12.136563+0000","last_fullsized":"2024-09-15T23:58:12.136563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:05.334727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148712+0000","last_change":"2024-09-15T23:58:11.150172+0000","last_active":"2024-09-15T23:58:12.148712+0000","last_peered":"2024-09-15T23:58:12.148712+0000","last_clean":"2024-09-15T23:58:12.148712+0000","last_became_active":"2024-09-15T23:58:11.149800+0000","last_became_peered":"2024-09-15T23:58:11.149800+0000","last_unstale":"2024-09-15T23:58:12.148712+0000","last_undegraded":"2024-09-15T23:58:12.148712+0000","last_fullsized":"2024-09-15T23:58:12.148712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:16.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154255+0000","last_change":"2024-09-15T23:58:11.153829+0000","last_active":"2024-09-15T23:58:12.154255+0000","last_peered":"2024-09-15T23:58:12.154255+0000","last_clean":"2024-09-15T23:58:12.154255+0000","last_became_active":"2024-09-15T23:58:11.153512+0000","last_became_peered":"2024-09-15T23:58:11.153512+0000","last_unstale":"2024-09-15T23:58:12.154255+0000","last_undegraded":"2024-09-15T23:58:12.154255+0000","last_fullsized":"2024-09-15T23:58:12.154255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:36.405204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147909+0000","last_change":"2024-09-15T23:58:11.162122+0000","last_active":"2024-09-15T23:58:12.147909+0000","last_peered":"2024-09-15T23:58:12.147909+0000","last_clean":"2024-09-15T23:58:12.147909+0000","last_became_active":"2024-09-15T23:58:11.161814+0000","last_became_peered":"2024-09-15T23:58:11.161814+0000","last_unstale":"2024-09-15T23:58:12.147909+0000","last_undegraded":"2024-09-15T23:58:12.147909+0000","last_fullsized":"2024-09-15T23:58:12.147909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:07.058458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150959+0000","last_change":"2024-09-15T23:58:11.157204+0000","last_active":"2024-09-15T23:58:12.150959+0000","last_peered":"2024-09-15T23:58:12.150959+0000","last_clean":"2024-09-15T23:58:12.150959+0000","last_became_active":"2024-09-15T23:58:11.156906+0000","last_became_peered":"2024-09-15T23:58:11.156906+0000","last_unstale":"2024-09-15T23:58:12.150959+0000","last_undegraded":"2024-09-15T23:58:12.150959+0000","last_fullsized":"2024-09-15T23:58:12.150959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:04.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148236+0000","last_change":"2024-09-15T23:58:11.150191+0000","last_active":"2024-09-15T23:58:12.148236+0000","last_peered":"2024-09-15T23:58:12.148236+0000","last_clean":"2024-09-15T23:58:12.148236+0000","last_became_active":"2024-09-15T23:58:11.149616+0000","last_became_peered":"2024-09-15T23:58:11.149616+0000","last_unstale":"2024-09-15T23:58:12.148236+0000","last_undegraded":"2024-09-15T23:58:12.148236+0000","last_fullsized":"2024-09-15T23:58:12.148236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:49.956626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148515+0000","last_change":"2024-09-15T23:58:11.162127+0000","last_active":"2024-09-15T23:58:12.148515+0000","last_peered":"2024-09-15T23:58:12.148515+0000","last_clean":"2024-09-15T23:58:12.148515+0000","last_became_active":"2024-09-15T23:58:11.161766+0000","last_became_peered":"2024-09-15T23:58:11.161766+0000","last_unstale":"2024-09-15T23:58:12.148515+0000","last_undegraded":"2024-09-15T23:58:12.148515+0000","last_fullsized":"2024-09-15T23:58:12.148515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:09.626747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154236+0000","last_change":"2024-09-15T23:58:11.153546+0000","last_active":"2024-09-15T23:58:12.154236+0000","last_peered":"2024-09-15T23:58:12.154236+0000","last_clean":"2024-09-15T23:58:12.154236+0000","last_became_active":"2024-09-15T23:58:11.153116+0000","last_became_peered":"2024-09-15T23:58:11.153116+0000","last_unstale":"2024-09-15T23:58:12.154236+0000","last_undegraded":"2024-09-15T23:58:12.154236+0000","last_fullsized":"2024-09-15T23:58:12.154236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:02.389068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148125+0000","last_change":"2024-09-15T23:58:11.147446+0000","last_active":"2024-09-15T23:58:12.148125+0000","last_peered":"2024-09-15T23:58:12.148125+0000","last_clean":"2024-09-15T23:58:12.148125+0000","last_became_active":"2024-09-15T23:58:11.146285+0000","last_became_peered":"2024-09-15T23:58:11.146285+0000","last_unstale":"2024-09-15T23:58:12.148125+0000","last_undegraded":"2024-09-15T23:58:12.148125+0000","last_fullsized":"2024-09-15T23:58:12.148125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:38.741814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":76,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:13.002001+0000","last_change":"2024-09-15T23:58:09.123719+0000","last_active":"2024-09-15T23:59:13.002001+0000","last_peered":"2024-09-15T23:59:13.002001+0000","last_clean":"2024-09-15T23:59:13.002001+0000","last_became_active":"2024-09-15T23:58:09.123082+0000","last_became_peered":"2024-09-15T23:58:09.123082+0000","last_unstale":"2024-09-15T23:59:13.002001+0000","last_undegraded":"2024-09-15T23:59:13.002001+0000","last_fullsized":"2024-09-15T23:59:13.002001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:46.704121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137195+0000","last_change":"2024-09-15T23:58:11.162571+0000","last_active":"2024-09-15T23:58:12.137195+0000","last_peered":"2024-09-15T23:58:12.137195+0000","last_clean":"2024-09-15T23:58:12.137195+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.137195+0000","last_undegraded":"2024-09-15T23:58:12.137195+0000","last_fullsized":"2024-09-15T23:58:12.137195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:58.134589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.290419+0000","last_change":"2024-09-15T23:58:09.125683+0000","last_active":"2024-09-15T23:58:22.290419+0000","last_peered":"2024-09-15T23:58:22.290419+0000","last_clean":"2024-09-15T23:58:22.290419+0000","last_became_active":"2024-09-15T23:58:09.125345+0000","last_became_peered":"2024-09-15T23:58:09.125345+0000","last_unstale":"2024-09-15T23:58:22.290419+0000","last_undegraded":"2024-09-15T23:58:22.290419+0000","last_fullsized":"2024-09-15T23:58:22.290419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:39.998850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284754+0000","last_change":"2024-09-15T23:58:09.116886+0000","last_active":"2024-09-15T23:58:21.284754+0000","last_peered":"2024-09-15T23:58:21.284754+0000","last_clean":"2024-09-15T23:58:21.284754+0000","last_became_active":"2024-09-15T23:58:09.116510+0000","last_became_peered":"2024-09-15T23:58:09.116510+0000","last_unstale":"2024-09-15T23:58:21.284754+0000","last_undegraded":"2024-09-15T23:58:21.284754+0000","last_fullsized":"2024-09-15T23:58:21.284754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:10.368106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136363+0000","last_change":"2024-09-15T23:58:11.144759+0000","last_active":"2024-09-15T23:58:12.136363+0000","last_peered":"2024-09-15T23:58:12.136363+0000","last_clean":"2024-09-15T23:58:12.136363+0000","last_became_active":"2024-09-15T23:58:11.144435+0000","last_became_peered":"2024-09-15T23:58:11.144435+0000","last_unstale":"2024-09-15T23:58:12.136363+0000","last_undegraded":"2024-09-15T23:58:12.136363+0000","last_fullsized":"2024-09-15T23:58:12.136363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:32.787727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300459+0000","last_change":"2024-09-15T23:58:09.124938+0000","last_active":"2024-09-15T23:58:23.300459+0000","last_peered":"2024-09-15T23:58:23.300459+0000","last_clean":"2024-09-15T23:58:23.300459+0000","last_became_active":"2024-09-15T23:58:09.124532+0000","last_became_peered":"2024-09-15T23:58:09.124532+0000","last_unstale":"2024-09-15T23:58:23.300459+0000","last_undegraded":"2024-09-15T23:58:23.300459+0000","last_fullsized":"2024-09-15T23:58:23.300459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148543+0000","last_change":"2024-09-15T23:58:11.145940+0000","last_active":"2024-09-15T23:58:12.148543+0000","last_peered":"2024-09-15T23:58:12.148543+0000","last_clean":"2024-09-15T23:58:12.148543+0000","last_became_active":"2024-09-15T23:58:11.145642+0000","last_became_peered":"2024-09-15T23:58:11.145642+0000","last_unstale":"2024-09-15T23:58:12.148543+0000","last_undegraded":"2024-09-15T23:58:12.148543+0000","last_fullsized":"2024-09-15T23:58:12.148543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:58.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:11.999977+0000","last_change":"2024-09-15T23:58:09.126742+0000","last_active":"2024-09-15T23:59:11.999977+0000","last_peered":"2024-09-15T23:59:11.999977+0000","last_clean":"2024-09-15T23:59:11.999977+0000","last_became_active":"2024-09-15T23:58:09.126432+0000","last_became_peered":"2024-09-15T23:58:09.126432+0000","last_unstale":"2024-09-15T23:59:11.999977+0000","last_undegraded":"2024-09-15T23:59:11.999977+0000","last_fullsized":"2024-09-15T23:59:11.999977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:11.582346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148667+0000","last_change":"2024-09-15T23:58:11.150130+0000","last_active":"2024-09-15T23:58:12.148667+0000","last_peered":"2024-09-15T23:58:12.148667+0000","last_clean":"2024-09-15T23:58:12.148667+0000","last_became_active":"2024-09-15T23:58:11.149531+0000","last_became_peered":"2024-09-15T23:58:11.149531+0000","last_unstale":"2024-09-15T23:58:12.148667+0000","last_undegraded":"2024-09-15T23:58:12.148667+0000","last_fullsized":"2024-09-15T23:58:12.148667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:07.206505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299397+0000","last_change":"2024-09-15T23:58:09.121229+0000","last_active":"2024-09-15T23:58:23.299397+0000","last_peered":"2024-09-15T23:58:23.299397+0000","last_clean":"2024-09-15T23:58:23.299397+0000","last_became_active":"2024-09-15T23:58:09.120808+0000","last_became_peered":"2024-09-15T23:58:09.120808+0000","last_unstale":"2024-09-15T23:58:23.299397+0000","last_undegraded":"2024-09-15T23:58:23.299397+0000","last_fullsized":"2024-09-15T23:58:23.299397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:34.155774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151252+0000","last_change":"2024-09-15T23:58:11.157253+0000","last_active":"2024-09-15T23:58:12.151252+0000","last_peered":"2024-09-15T23:58:12.151252+0000","last_clean":"2024-09-15T23:58:12.151252+0000","last_became_active":"2024-09-15T23:58:11.156952+0000","last_became_peered":"2024-09-15T23:58:11.156952+0000","last_unstale":"2024-09-15T23:58:12.151252+0000","last_undegraded":"2024-09-15T23:58:12.151252+0000","last_fullsized":"2024-09-15T23:58:12.151252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:09.840371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289729+0000","last_change":"2024-09-15T23:58:09.322776+0000","last_active":"2024-09-15T23:58:22.289729+0000","last_peered":"2024-09-15T23:58:22.289729+0000","last_clean":"2024-09-15T23:58:22.289729+0000","last_became_active":"2024-09-15T23:58:09.322304+0000","last_became_peered":"2024-09-15T23:58:09.322304+0000","last_unstale":"2024-09-15T23:58:22.289729+0000","last_undegraded":"2024-09-15T23:58:22.289729+0000","last_fullsized":"2024-09-15T23:58:22.289729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:51.413578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148251+0000","last_change":"2024-09-15T23:58:11.165067+0000","last_active":"2024-09-15T23:58:12.148251+0000","last_peered":"2024-09-15T23:58:12.148251+0000","last_clean":"2024-09-15T23:58:12.148251+0000","last_became_active":"2024-09-15T23:58:11.163722+0000","last_became_peered":"2024-09-15T23:58:11.163722+0000","last_unstale":"2024-09-15T23:58:12.148251+0000","last_undegraded":"2024-09-15T23:58:12.148251+0000","last_fullsized":"2024-09-15T23:58:12.148251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:01.354822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:11.999839+0000","last_change":"2024-09-15T23:58:09.128077+0000","last_active":"2024-09-15T23:59:11.999839+0000","last_peered":"2024-09-15T23:59:11.999839+0000","last_clean":"2024-09-15T23:59:11.999839+0000","last_became_active":"2024-09-15T23:58:09.127215+0000","last_became_peered":"2024-09-15T23:58:09.127215+0000","last_unstale":"2024-09-15T23:59:11.999839+0000","last_undegraded":"2024-09-15T23:59:11.999839+0000","last_fullsized":"2024-09-15T23:59:11.999839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:47.875279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":113,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150490+0000","last_change":"2024-09-15T23:58:11.156020+0000","last_active":"2024-09-15T23:58:12.150490+0000","last_peered":"2024-09-15T23:58:12.150490+0000","last_clean":"2024-09-15T23:58:12.150490+0000","last_became_active":"2024-09-15T23:58:11.155634+0000","last_became_peered":"2024-09-15T23:58:11.155634+0000","last_unstale":"2024-09-15T23:58:12.150490+0000","last_undegraded":"2024-09-15T23:58:12.150490+0000","last_fullsized":"2024-09-15T23:58:12.150490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934453+0000","last_change":"2024-09-15T23:58:09.123084+0000","last_active":"2024-09-15T23:58:23.934453+0000","last_peered":"2024-09-15T23:58:23.934453+0000","last_clean":"2024-09-15T23:58:23.934453+0000","last_became_active":"2024-09-15T23:58:09.122594+0000","last_became_peered":"2024-09-15T23:58:09.122594+0000","last_unstale":"2024-09-15T23:58:23.934453+0000","last_undegraded":"2024-09-15T23:58:23.934453+0000","last_fullsized":"2024-09-15T23:58:23.934453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:47.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149295+0000","last_change":"2024-09-15T23:58:11.155082+0000","last_active":"2024-09-15T23:58:12.149295+0000","last_peered":"2024-09-15T23:58:12.149295+0000","last_clean":"2024-09-15T23:58:12.149295+0000","last_became_active":"2024-09-15T23:58:11.154764+0000","last_became_peered":"2024-09-15T23:58:11.154764+0000","last_unstale":"2024-09-15T23:58:12.149295+0000","last_undegraded":"2024-09-15T23:58:12.149295+0000","last_fullsized":"2024-09-15T23:58:12.149295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:47.170383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300267+0000","last_change":"2024-09-15T23:58:09.124416+0000","last_active":"2024-09-15T23:58:23.300267+0000","last_peered":"2024-09-15T23:58:23.300267+0000","last_clean":"2024-09-15T23:58:23.300267+0000","last_became_active":"2024-09-15T23:58:09.123836+0000","last_became_peered":"2024-09-15T23:58:09.123836+0000","last_unstale":"2024-09-15T23:58:23.300267+0000","last_undegraded":"2024-09-15T23:58:23.300267+0000","last_fullsized":"2024-09-15T23:58:23.300267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:35.730677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150934+0000","last_change":"2024-09-15T23:58:11.162220+0000","last_active":"2024-09-15T23:58:12.150934+0000","last_peered":"2024-09-15T23:58:12.150934+0000","last_clean":"2024-09-15T23:58:12.150934+0000","last_became_active":"2024-09-15T23:58:11.161887+0000","last_became_peered":"2024-09-15T23:58:11.161887+0000","last_unstale":"2024-09-15T23:58:12.150934+0000","last_undegraded":"2024-09-15T23:58:12.150934+0000","last_fullsized":"2024-09-15T23:58:12.150934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:35.070135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:14.002832+0000","last_change":"2024-09-15T23:58:09.123010+0000","last_active":"2024-09-15T23:59:14.002832+0000","last_peered":"2024-09-15T23:59:14.002832+0000","last_clean":"2024-09-15T23:59:14.002832+0000","last_became_active":"2024-09-15T23:58:09.122615+0000","last_became_peered":"2024-09-15T23:58:09.122615+0000","last_unstale":"2024-09-15T23:59:14.002832+0000","last_undegraded":"2024-09-15T23:59:14.002832+0000","last_fullsized":"2024-09-15T23:59:14.002832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:09.634409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150261+0000","last_change":"2024-09-15T23:58:11.159069+0000","last_active":"2024-09-15T23:58:12.150261+0000","last_peered":"2024-09-15T23:58:12.150261+0000","last_clean":"2024-09-15T23:58:12.150261+0000","last_became_active":"2024-09-15T23:58:11.158659+0000","last_became_peered":"2024-09-15T23:58:11.158659+0000","last_unstale":"2024-09-15T23:58:12.150261+0000","last_undegraded":"2024-09-15T23:58:12.150261+0000","last_fullsized":"2024-09-15T23:58:12.150261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:24.336089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298401+0000","last_change":"2024-09-15T23:58:09.322903+0000","last_active":"2024-09-15T23:58:23.298401+0000","last_peered":"2024-09-15T23:58:23.298401+0000","last_clean":"2024-09-15T23:58:23.298401+0000","last_became_active":"2024-09-15T23:58:09.322326+0000","last_became_peered":"2024-09-15T23:58:09.322326+0000","last_unstale":"2024-09-15T23:58:23.298401+0000","last_undegraded":"2024-09-15T23:58:23.298401+0000","last_fullsized":"2024-09-15T23:58:23.298401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:58.749293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136919+0000","last_change":"2024-09-15T23:58:11.162104+0000","last_active":"2024-09-15T23:58:12.136919+0000","last_peered":"2024-09-15T23:58:12.136919+0000","last_clean":"2024-09-15T23:58:12.136919+0000","last_became_active":"2024-09-15T23:58:11.161739+0000","last_became_peered":"2024-09-15T23:58:11.161739+0000","last_unstale":"2024-09-15T23:58:12.136919+0000","last_undegraded":"2024-09-15T23:58:12.136919+0000","last_fullsized":"2024-09-15T23:58:12.136919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:55.316245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.279815+0000","last_change":"2024-09-15T23:58:09.119975+0000","last_active":"2024-09-15T23:58:22.279815+0000","last_peered":"2024-09-15T23:58:22.279815+0000","last_clean":"2024-09-15T23:58:22.279815+0000","last_became_active":"2024-09-15T23:58:09.119340+0000","last_became_peered":"2024-09-15T23:58:09.119340+0000","last_unstale":"2024-09-15T23:58:22.279815+0000","last_undegraded":"2024-09-15T23:58:22.279815+0000","last_fullsized":"2024-09-15T23:58:22.279815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:22:14.858076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153874+0000","last_change":"2024-09-15T23:58:11.157389+0000","last_active":"2024-09-15T23:58:12.153874+0000","last_peered":"2024-09-15T23:58:12.153874+0000","last_clean":"2024-09-15T23:58:12.153874+0000","last_became_active":"2024-09-15T23:58:11.157033+0000","last_became_peered":"2024-09-15T23:58:11.157033+0000","last_unstale":"2024-09-15T23:58:12.153874+0000","last_undegraded":"2024-09-15T23:58:12.153874+0000","last_fullsized":"2024-09-15T23:58:12.153874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:49.149860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299541+0000","last_change":"2024-09-15T23:58:09.120937+0000","last_active":"2024-09-15T23:58:23.299541+0000","last_peered":"2024-09-15T23:58:23.299541+0000","last_clean":"2024-09-15T23:58:23.299541+0000","last_became_active":"2024-09-15T23:58:09.120586+0000","last_became_peered":"2024-09-15T23:58:09.120586+0000","last_unstale":"2024-09-15T23:58:23.299541+0000","last_undegraded":"2024-09-15T23:58:23.299541+0000","last_fullsized":"2024-09-15T23:58:23.299541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:18.432631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150943+0000","last_change":"2024-09-15T23:58:11.155316+0000","last_active":"2024-09-15T23:58:12.150943+0000","last_peered":"2024-09-15T23:58:12.150943+0000","last_clean":"2024-09-15T23:58:12.150943+0000","last_became_active":"2024-09-15T23:58:11.154975+0000","last_became_peered":"2024-09-15T23:58:11.154975+0000","last_unstale":"2024-09-15T23:58:12.150943+0000","last_undegraded":"2024-09-15T23:58:12.150943+0000","last_fullsized":"2024-09-15T23:58:12.150943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:09.856126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300569+0000","last_change":"2024-09-15T23:58:09.125045+0000","last_active":"2024-09-15T23:58:23.300569+0000","last_peered":"2024-09-15T23:58:23.300569+0000","last_clean":"2024-09-15T23:58:23.300569+0000","last_became_active":"2024-09-15T23:58:09.124577+0000","last_became_peered":"2024-09-15T23:58:09.124577+0000","last_unstale":"2024-09-15T23:58:23.300569+0000","last_undegraded":"2024-09-15T23:58:23.300569+0000","last_fullsized":"2024-09-15T23:58:23.300569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:48.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150843+0000","last_change":"2024-09-15T23:58:11.160241+0000","last_active":"2024-09-15T23:58:12.150843+0000","last_peered":"2024-09-15T23:58:12.150843+0000","last_clean":"2024-09-15T23:58:12.150843+0000","last_became_active":"2024-09-15T23:58:11.159886+0000","last_became_peered":"2024-09-15T23:58:11.159886+0000","last_unstale":"2024-09-15T23:58:12.150843+0000","last_undegraded":"2024-09-15T23:58:12.150843+0000","last_fullsized":"2024-09-15T23:58:12.150843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:02.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:11.009295+0000","last_change":"2024-09-15T23:58:09.482888+0000","last_active":"2024-09-15T23:59:11.009295+0000","last_peered":"2024-09-15T23:59:11.009295+0000","last_clean":"2024-09-15T23:59:11.009295+0000","last_became_active":"2024-09-15T23:58:09.482497+0000","last_became_peered":"2024-09-15T23:58:09.482497+0000","last_unstale":"2024-09-15T23:59:11.009295+0000","last_undegraded":"2024-09-15T23:59:11.009295+0000","last_fullsized":"2024-09-15T23:59:11.009295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:59.320794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151266+0000","last_change":"2024-09-15T23:58:11.159452+0000","last_active":"2024-09-15T23:58:12.151266+0000","last_peered":"2024-09-15T23:58:12.151266+0000","last_clean":"2024-09-15T23:58:12.151266+0000","last_became_active":"2024-09-15T23:58:11.157610+0000","last_became_peered":"2024-09-15T23:58:11.157610+0000","last_unstale":"2024-09-15T23:58:12.151266+0000","last_undegraded":"2024-09-15T23:58:12.151266+0000","last_fullsized":"2024-09-15T23:58:12.151266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:17.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150373+0000","last_change":"2024-09-15T23:56:56.876989+0000","last_active":"2024-09-15T23:58:12.150373+0000","last_peered":"2024-09-15T23:58:12.150373+0000","last_clean":"2024-09-15T23:58:12.150373+0000","last_became_active":"2024-09-15T23:56:56.876413+0000","last_became_peered":"2024-09-15T23:56:56.876413+0000","last_unstale":"2024-09-15T23:58:12.150373+0000","last_undegraded":"2024-09-15T23:58:12.150373+0000","last_fullsized":"2024-09-15T23:58:12.150373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_clean_scrub_stamp":"2024-09-15T23:53:27.674229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:06.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934257+0000","last_change":"2024-09-15T23:58:09.124514+0000","last_active":"2024-09-15T23:58:23.934257+0000","last_peered":"2024-09-15T23:58:23.934257+0000","last_clean":"2024-09-15T23:58:23.934257+0000","last_became_active":"2024-09-15T23:58:09.122124+0000","last_became_peered":"2024-09-15T23:58:09.122124+0000","last_unstale":"2024-09-15T23:58:23.934257+0000","last_undegraded":"2024-09-15T23:58:23.934257+0000","last_fullsized":"2024-09-15T23:58:23.934257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:13.563073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136568+0000","last_change":"2024-09-15T23:58:11.149559+0000","last_active":"2024-09-15T23:58:12.136568+0000","last_peered":"2024-09-15T23:58:12.136568+0000","last_clean":"2024-09-15T23:58:12.136568+0000","last_became_active":"2024-09-15T23:58:11.149215+0000","last_became_peered":"2024-09-15T23:58:11.149215+0000","last_unstale":"2024-09-15T23:58:12.136568+0000","last_undegraded":"2024-09-15T23:58:12.136568+0000","last_fullsized":"2024-09-15T23:58:12.136568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:16.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284817+0000","last_change":"2024-09-15T23:58:09.127770+0000","last_active":"2024-09-15T23:58:21.284817+0000","last_peered":"2024-09-15T23:58:21.284817+0000","last_clean":"2024-09-15T23:58:21.284817+0000","last_became_active":"2024-09-15T23:58:09.127404+0000","last_became_peered":"2024-09-15T23:58:09.127404+0000","last_unstale":"2024-09-15T23:58:21.284817+0000","last_undegraded":"2024-09-15T23:58:21.284817+0000","last_fullsized":"2024-09-15T23:58:21.284817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:48.176290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148872+0000","last_change":"2024-09-15T23:58:11.162161+0000","last_active":"2024-09-15T23:58:12.148872+0000","last_peered":"2024-09-15T23:58:12.148872+0000","last_clean":"2024-09-15T23:58:12.148872+0000","last_became_active":"2024-09-15T23:58:11.161879+0000","last_became_peered":"2024-09-15T23:58:11.161879+0000","last_unstale":"2024-09-15T23:58:12.148872+0000","last_undegraded":"2024-09-15T23:58:12.148872+0000","last_fullsized":"2024-09-15T23:58:12.148872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:15.003851+0000","last_change":"2024-09-15T23:58:09.322611+0000","last_active":"2024-09-15T23:59:15.003851+0000","last_peered":"2024-09-15T23:59:15.003851+0000","last_clean":"2024-09-15T23:59:15.003851+0000","last_became_active":"2024-09-15T23:58:09.322254+0000","last_became_peered":"2024-09-15T23:58:09.322254+0000","last_unstale":"2024-09-15T23:59:15.003851+0000","last_undegraded":"2024-09-15T23:59:15.003851+0000","last_fullsized":"2024-09-15T23:59:15.003851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:03.803927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137151+0000","last_change":"2024-09-15T23:58:11.152219+0000","last_active":"2024-09-15T23:58:12.137151+0000","last_peered":"2024-09-15T23:58:12.137151+0000","last_clean":"2024-09-15T23:58:12.137151+0000","last_became_active":"2024-09-15T23:58:11.151885+0000","last_became_peered":"2024-09-15T23:58:11.151885+0000","last_unstale":"2024-09-15T23:58:12.137151+0000","last_undegraded":"2024-09-15T23:58:12.137151+0000","last_fullsized":"2024-09-15T23:58:12.137151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:10.780163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:15.014278+0000","last_change":"2024-09-15T23:58:09.125152+0000","last_active":"2024-09-15T23:59:15.014278+0000","last_peered":"2024-09-15T23:59:15.014278+0000","last_clean":"2024-09-15T23:59:15.014278+0000","last_became_active":"2024-09-15T23:58:09.124754+0000","last_became_peered":"2024-09-15T23:58:09.124754+0000","last_unstale":"2024-09-15T23:59:15.014278+0000","last_undegraded":"2024-09-15T23:59:15.014278+0000","last_fullsized":"2024-09-15T23:59:15.014278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:02.094909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136921+0000","last_change":"2024-09-15T23:58:11.160257+0000","last_active":"2024-09-15T23:58:12.136921+0000","last_peered":"2024-09-15T23:58:12.136921+0000","last_clean":"2024-09-15T23:58:12.136921+0000","last_became_active":"2024-09-15T23:58:11.159868+0000","last_became_peered":"2024-09-15T23:58:11.159868+0000","last_unstale":"2024-09-15T23:58:12.136921+0000","last_undegraded":"2024-09-15T23:58:12.136921+0000","last_fullsized":"2024-09-15T23:58:12.136921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.528002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.226786+0000","last_change":"2024-09-15T23:58:09.323034+0000","last_active":"2024-09-15T23:58:12.226786+0000","last_peered":"2024-09-15T23:58:12.226786+0000","last_clean":"2024-09-15T23:58:12.226786+0000","last_became_active":"2024-09-15T23:58:09.322658+0000","last_became_peered":"2024-09-15T23:58:09.322658+0000","last_unstale":"2024-09-15T23:58:12.226786+0000","last_undegraded":"2024-09-15T23:58:12.226786+0000","last_fullsized":"2024-09-15T23:58:12.226786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:20:11.175093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.862670+0000","last_change":"2024-09-15T23:58:11.149216+0000","last_active":"2024-09-15T23:58:12.862670+0000","last_peered":"2024-09-15T23:58:12.862670+0000","last_clean":"2024-09-15T23:58:12.862670+0000","last_became_active":"2024-09-15T23:58:11.148893+0000","last_became_peered":"2024-09-15T23:58:11.148893+0000","last_unstale":"2024-09-15T23:58:12.862670+0000","last_undegraded":"2024-09-15T23:58:12.862670+0000","last_fullsized":"2024-09-15T23:58:12.862670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:07.749101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300019+0000","last_change":"2024-09-15T23:58:09.323030+0000","last_active":"2024-09-15T23:58:23.300019+0000","last_peered":"2024-09-15T23:58:23.300019+0000","last_clean":"2024-09-15T23:58:23.300019+0000","last_became_active":"2024-09-15T23:58:09.322569+0000","last_became_peered":"2024-09-15T23:58:09.322569+0000","last_unstale":"2024-09-15T23:58:23.300019+0000","last_undegraded":"2024-09-15T23:58:23.300019+0000","last_fullsized":"2024-09-15T23:58:23.300019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:52.811395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148063+0000","last_change":"2024-09-15T23:58:11.154084+0000","last_active":"2024-09-15T23:58:12.148063+0000","last_peered":"2024-09-15T23:58:12.148063+0000","last_clean":"2024-09-15T23:58:12.148063+0000","last_became_active":"2024-09-15T23:58:11.153720+0000","last_became_peered":"2024-09-15T23:58:11.153720+0000","last_unstale":"2024-09-15T23:58:12.148063+0000","last_undegraded":"2024-09-15T23:58:12.148063+0000","last_fullsized":"2024-09-15T23:58:12.148063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:07.468812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:38.149932+0000","last_change":"2024-09-15T23:58:09.116996+0000","last_active":"2024-09-15T23:58:38.149932+0000","last_peered":"2024-09-15T23:58:38.149932+0000","last_clean":"2024-09-15T23:58:38.149932+0000","last_became_active":"2024-09-15T23:58:09.116531+0000","last_became_peered":"2024-09-15T23:58:09.116531+0000","last_unstale":"2024-09-15T23:58:38.149932+0000","last_undegraded":"2024-09-15T23:58:38.149932+0000","last_fullsized":"2024-09-15T23:58:38.149932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.094719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136595+0000","last_change":"2024-09-15T23:58:11.155704+0000","last_active":"2024-09-15T23:58:12.136595+0000","last_peered":"2024-09-15T23:58:12.136595+0000","last_clean":"2024-09-15T23:58:12.136595+0000","last_became_active":"2024-09-15T23:58:11.155344+0000","last_became_peered":"2024-09-15T23:58:11.155344+0000","last_unstale":"2024-09-15T23:58:12.136595+0000","last_undegraded":"2024-09-15T23:58:12.136595+0000","last_fullsized":"2024-09-15T23:58:12.136595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:50.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298948+0000","last_change":"2024-09-15T23:58:09.482837+0000","last_active":"2024-09-15T23:58:23.298948+0000","last_peered":"2024-09-15T23:58:23.298948+0000","last_clean":"2024-09-15T23:58:23.298948+0000","last_became_active":"2024-09-15T23:58:09.482444+0000","last_became_peered":"2024-09-15T23:58:09.482444+0000","last_unstale":"2024-09-15T23:58:23.298948+0000","last_undegraded":"2024-09-15T23:58:23.298948+0000","last_fullsized":"2024-09-15T23:58:23.298948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:05.962471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150939+0000","last_change":"2024-09-15T23:58:11.157249+0000","last_active":"2024-09-15T23:58:12.150939+0000","last_peered":"2024-09-15T23:58:12.150939+0000","last_clean":"2024-09-15T23:58:12.150939+0000","last_became_active":"2024-09-15T23:58:11.156871+0000","last_became_peered":"2024-09-15T23:58:11.156871+0000","last_unstale":"2024-09-15T23:58:12.150939+0000","last_undegraded":"2024-09-15T23:58:12.150939+0000","last_fullsized":"2024-09-15T23:58:12.150939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:16.986165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279567+0000","last_change":"2024-09-15T23:58:09.324320+0000","last_active":"2024-09-15T23:58:20.279567+0000","last_peered":"2024-09-15T23:58:20.279567+0000","last_clean":"2024-09-15T23:58:20.279567+0000","last_became_active":"2024-09-15T23:58:09.323996+0000","last_became_peered":"2024-09-15T23:58:09.323996+0000","last_unstale":"2024-09-15T23:58:20.279567+0000","last_undegraded":"2024-09-15T23:58:20.279567+0000","last_fullsized":"2024-09-15T23:58:20.279567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:34.456229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147904+0000","last_change":"2024-09-15T23:58:11.146836+0000","last_active":"2024-09-15T23:58:12.147904+0000","last_peered":"2024-09-15T23:58:12.147904+0000","last_clean":"2024-09-15T23:58:12.147904+0000","last_became_active":"2024-09-15T23:58:11.146470+0000","last_became_peered":"2024-09-15T23:58:11.146470+0000","last_unstale":"2024-09-15T23:58:12.147904+0000","last_undegraded":"2024-09-15T23:58:12.147904+0000","last_fullsized":"2024-09-15T23:58:12.147904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.721954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300504+0000","last_change":"2024-09-15T23:58:09.121920+0000","last_active":"2024-09-15T23:58:23.300504+0000","last_peered":"2024-09-15T23:58:23.300504+0000","last_clean":"2024-09-15T23:58:23.300504+0000","last_became_active":"2024-09-15T23:58:09.121605+0000","last_became_peered":"2024-09-15T23:58:09.121605+0000","last_unstale":"2024-09-15T23:58:23.300504+0000","last_undegraded":"2024-09-15T23:58:23.300504+0000","last_fullsized":"2024-09-15T23:58:23.300504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:12.707690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154389+0000","last_change":"2024-09-15T23:58:11.153676+0000","last_active":"2024-09-15T23:58:12.154389+0000","last_peered":"2024-09-15T23:58:12.154389+0000","last_clean":"2024-09-15T23:58:12.154389+0000","last_became_active":"2024-09-15T23:58:11.152866+0000","last_became_peered":"2024-09-15T23:58:11.152866+0000","last_unstale":"2024-09-15T23:58:12.154389+0000","last_undegraded":"2024-09-15T23:58:12.154389+0000","last_fullsized":"2024-09-15T23:58:12.154389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:02.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289656+0000","last_change":"2024-09-15T23:58:09.121440+0000","last_active":"2024-09-15T23:58:22.289656+0000","last_peered":"2024-09-15T23:58:22.289656+0000","last_clean":"2024-09-15T23:58:22.289656+0000","last_became_active":"2024-09-15T23:58:09.121006+0000","last_became_peered":"2024-09-15T23:58:09.121006+0000","last_unstale":"2024-09-15T23:58:22.289656+0000","last_undegraded":"2024-09-15T23:58:22.289656+0000","last_fullsized":"2024-09-15T23:58:22.289656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:32.584222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136632+0000","last_change":"2024-09-15T23:58:11.155311+0000","last_active":"2024-09-15T23:58:12.136632+0000","last_peered":"2024-09-15T23:58:12.136632+0000","last_clean":"2024-09-15T23:58:12.136632+0000","last_became_active":"2024-09-15T23:58:11.154954+0000","last_became_peered":"2024-09-15T23:58:11.154954+0000","last_unstale":"2024-09-15T23:58:12.136632+0000","last_undegraded":"2024-09-15T23:58:12.136632+0000","last_fullsized":"2024-09-15T23:58:12.136632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:52.131451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279891+0000","last_change":"2024-09-15T23:58:09.126143+0000","last_active":"2024-09-15T23:58:20.279891+0000","last_peered":"2024-09-15T23:58:20.279891+0000","last_clean":"2024-09-15T23:58:20.279891+0000","last_became_active":"2024-09-15T23:58:09.125785+0000","last_became_peered":"2024-09-15T23:58:09.125785+0000","last_unstale":"2024-09-15T23:58:20.279891+0000","last_undegraded":"2024-09-15T23:58:20.279891+0000","last_fullsized":"2024-09-15T23:58:20.279891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:22.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150690+0000","last_change":"2024-09-15T23:58:11.160834+0000","last_active":"2024-09-15T23:58:12.150690+0000","last_peered":"2024-09-15T23:58:12.150690+0000","last_clean":"2024-09-15T23:58:12.150690+0000","last_became_active":"2024-09-15T23:58:11.160458+0000","last_became_peered":"2024-09-15T23:58:11.160458+0000","last_unstale":"2024-09-15T23:58:12.150690+0000","last_undegraded":"2024-09-15T23:58:12.150690+0000","last_fullsized":"2024-09-15T23:58:12.150690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:36.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137347+0000","last_change":"2024-09-15T23:58:11.151745+0000","last_active":"2024-09-15T23:58:12.137347+0000","last_peered":"2024-09-15T23:58:12.137347+0000","last_clean":"2024-09-15T23:58:12.137347+0000","last_became_active":"2024-09-15T23:58:11.150864+0000","last_became_peered":"2024-09-15T23:58:11.150864+0000","last_unstale":"2024-09-15T23:58:12.137347+0000","last_undegraded":"2024-09-15T23:58:12.137347+0000","last_fullsized":"2024-09-15T23:58:12.137347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:34.578125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300990+0000","last_change":"2024-09-15T23:58:09.121095+0000","last_active":"2024-09-15T23:58:23.300990+0000","last_peered":"2024-09-15T23:58:23.300990+0000","last_clean":"2024-09-15T23:58:23.300990+0000","last_became_active":"2024-09-15T23:58:09.120671+0000","last_became_peered":"2024-09-15T23:58:09.120671+0000","last_unstale":"2024-09-15T23:58:23.300990+0000","last_undegraded":"2024-09-15T23:58:23.300990+0000","last_fullsized":"2024-09-15T23:58:23.300990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:32.099810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148041+0000","last_change":"2024-09-15T23:58:11.165256+0000","last_active":"2024-09-15T23:58:12.148041+0000","last_peered":"2024-09-15T23:58:12.148041+0000","last_clean":"2024-09-15T23:58:12.148041+0000","last_became_active":"2024-09-15T23:58:11.163663+0000","last_became_peered":"2024-09-15T23:58:11.163663+0000","last_unstale":"2024-09-15T23:58:12.148041+0000","last_undegraded":"2024-09-15T23:58:12.148041+0000","last_fullsized":"2024-09-15T23:58:12.148041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:02.112109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934080+0000","last_change":"2024-09-15T23:58:09.125833+0000","last_active":"2024-09-15T23:58:23.934080+0000","last_peered":"2024-09-15T23:58:23.934080+0000","last_clean":"2024-09-15T23:58:23.934080+0000","last_became_active":"2024-09-15T23:58:09.125411+0000","last_became_peered":"2024-09-15T23:58:09.125411+0000","last_unstale":"2024-09-15T23:58:23.934080+0000","last_undegraded":"2024-09-15T23:58:23.934080+0000","last_fullsized":"2024-09-15T23:58:23.934080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:17.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151115+0000","last_change":"2024-09-15T23:58:11.159923+0000","last_active":"2024-09-15T23:58:12.151115+0000","last_peered":"2024-09-15T23:58:12.151115+0000","last_clean":"2024-09-15T23:58:12.151115+0000","last_became_active":"2024-09-15T23:58:11.159489+0000","last_became_peered":"2024-09-15T23:58:11.159489+0000","last_unstale":"2024-09-15T23:58:12.151115+0000","last_undegraded":"2024-09-15T23:58:12.151115+0000","last_fullsized":"2024-09-15T23:58:12.151115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:30.864728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153095+0000","last_change":"2024-09-15T23:58:09.322767+0000","last_active":"2024-09-15T23:58:12.153095+0000","last_peered":"2024-09-15T23:58:12.153095+0000","last_clean":"2024-09-15T23:58:12.153095+0000","last_became_active":"2024-09-15T23:58:09.322289+0000","last_became_peered":"2024-09-15T23:58:09.322289+0000","last_unstale":"2024-09-15T23:58:12.153095+0000","last_undegraded":"2024-09-15T23:58:12.153095+0000","last_fullsized":"2024-09-15T23:58:12.153095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:12.168772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136810+0000","last_change":"2024-09-15T23:58:11.153181+0000","last_active":"2024-09-15T23:58:12.136810+0000","last_peered":"2024-09-15T23:58:12.136810+0000","last_clean":"2024-09-15T23:58:12.136810+0000","last_became_active":"2024-09-15T23:58:11.152920+0000","last_became_peered":"2024-09-15T23:58:11.152920+0000","last_unstale":"2024-09-15T23:58:12.136810+0000","last_undegraded":"2024-09-15T23:58:12.136810+0000","last_fullsized":"2024-09-15T23:58:12.136810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:15.101157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":81,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:11.008122+0000","last_change":"2024-09-15T23:58:09.120470+0000","last_active":"2024-09-15T23:59:11.008122+0000","last_peered":"2024-09-15T23:59:11.008122+0000","last_clean":"2024-09-15T23:59:11.008122+0000","last_became_active":"2024-09-15T23:58:09.120143+0000","last_became_peered":"2024-09-15T23:58:09.120143+0000","last_unstale":"2024-09-15T23:59:11.008122+0000","last_undegraded":"2024-09-15T23:59:11.008122+0000","last_fullsized":"2024-09-15T23:59:11.008122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:41.285326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68567,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":542,"num_read_kb":344,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":205701,"data_compressed":5418,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244980,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1070174,"data_compressed":37506,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":417323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1010221,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1009506,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":416865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":479925,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":480098,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":418379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63897,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63724,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63488,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:59:19.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T23:59:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:19 smithi102 ceph-mon[34458]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:19 smithi031 ceph-mon[31203]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:19 smithi081 ceph-mon[35060]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2978823776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84102ef0 msgr2=0x7f4c8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2978823776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84102ef0 0x7f4c8410da40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009ca0 tx=0x7f4c7402a950 comp rx=0 tx=0).stop 2024-09-15T23:59:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2978823776 shutdown_connections 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2978823776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84102ef0 0x7f4c8410da40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2978823776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c84102570 0x7f4c841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2978823776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84100e20 0x7f4c84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2978823776 >> 172.21.15.31:0/2978823776 conn(0x7f4c840fc7d0 msgr2=0x7f4c840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2978823776 shutdown_connections 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2978823776 wait complete. 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 Processor -- start 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.998+0000 7f4c8c07c640 1 -- start start 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 0x7f4c841996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c84102ef0 0x7f4c841a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84110e30 con 0x7f4c84100e20 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84110cb0 con 0x7f4c84102570 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c8c07c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84110fb0 con 0x7f4c84102ef0 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34914/0 (socket says 172.21.15.31:34914) 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 -- 172.21.15.31:0/2156683906 learned_addr learned my addr 172.21.15.31:0/2156683906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89617640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 0x7f4c841996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 -- 172.21.15.31:0/2156683906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c84102ef0 msgr2=0x7f4c841a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c84102ef0 0x7f4c841a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 -- 172.21.15.31:0/2156683906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 msgr2=0x7f4c841996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 0x7f4c841996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 -- 172.21.15.31:0/2156683906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c841a4740 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89617640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 0x7f4c841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c89e18640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:19.999+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c017070 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.000+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c7c009c30 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.000+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c841a4910 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.000+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00f620 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.000+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c841a4e50 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.001+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c84101200 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.002+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4c7c0109a0 con 0x7f4c84100e20 2024-09-15T23:59:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.002+0000 7f4c82ffd640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 0x7f4c6c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.002+0000 7f4c89617640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 0x7f4c6c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.003+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4c7c0cf570 con 0x7f4c84100e20 2024-09-15T23:59:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.003+0000 7f4c89617640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 0x7f4c6c0a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c78005e10 tx=0x7f4c78005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.007+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7c009de0 con 0x7f4c84100e20 2024-09-15T23:59:20.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.182+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c84106f50 con 0x7f4c6c0a3350 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:59:20.183 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:20 smithi031 ceph-mon[31203]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.188+0000 7f4c82ffd640 1 -- 172.21.15.31:0/2156683906 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258943 (secure 0 0 0) 0x7f4c84106f50 con 0x7f4c6c0a3350 2024-09-15T23:59:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-15T23:59:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 msgr2=0x7f4c6c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 0x7f4c6c0a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c78005e10 tx=0x7f4c78005da0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 msgr2=0x7f4c84199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00de10 comp rx=0 tx=0).stop 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 shutdown_connections 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f4c6c0a3350 0x7f4c6c0a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c84102ef0 0x7f4c841a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c84102570 0x7f4c841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 --2- 172.21.15.31:0/2156683906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c84100e20 0x7f4c84199180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 >> 172.21.15.31:0/2156683906 conn(0x7f4c840fc7d0 msgr2=0x7f4c84106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 shutdown_connections 2024-09-15T23:59:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:20.192+0000 7f4c8c07c640 1 -- 172.21.15.31:0/2156683906 wait complete. 2024-09-15T23:59:20.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:59:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:59:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:20 smithi102 ceph-mon[34458]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:59:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:59:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:20 smithi081 ceph-mon[35060]: from='mgr.14150 172.21.15.31:0/3693632779' entity='mgr.x' 2024-09-15T23:59:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-15T23:59:19.215831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659169,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":708,"num_read_kb":483,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347556,"kb_used_data":10788,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594268,"statfs":{"total":1151940427776,"available":1151584530432,"internally_reserved":0,"allocated":11046912,"data_stored":6971919,"data_compressed":112518,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":344634682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002867"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154743+0000","last_change":"2024-09-15T23:58:11.160319+0000","last_active":"2024-09-15T23:58:12.154743+0000","last_peered":"2024-09-15T23:58:12.154743+0000","last_clean":"2024-09-15T23:58:12.154743+0000","last_became_active":"2024-09-15T23:58:11.159572+0000","last_became_peered":"2024-09-15T23:58:11.159572+0000","last_unstale":"2024-09-15T23:58:12.154743+0000","last_undegraded":"2024-09-15T23:58:12.154743+0000","last_fullsized":"2024-09-15T23:58:12.154743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:50.598393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150768+0000","last_change":"2024-09-15T23:58:11.152044+0000","last_active":"2024-09-15T23:58:12.150768+0000","last_peered":"2024-09-15T23:58:12.150768+0000","last_clean":"2024-09-15T23:58:12.150768+0000","last_became_active":"2024-09-15T23:58:11.151472+0000","last_became_peered":"2024-09-15T23:58:11.151472+0000","last_unstale":"2024-09-15T23:58:12.150768+0000","last_undegraded":"2024-09-15T23:58:12.150768+0000","last_fullsized":"2024-09-15T23:58:12.150768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:51.502165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150571+0000","last_change":"2024-09-15T23:58:11.151854+0000","last_active":"2024-09-15T23:58:12.150571+0000","last_peered":"2024-09-15T23:58:12.150571+0000","last_clean":"2024-09-15T23:58:12.150571+0000","last_became_active":"2024-09-15T23:58:11.151418+0000","last_became_peered":"2024-09-15T23:58:11.151418+0000","last_unstale":"2024-09-15T23:58:12.150571+0000","last_undegraded":"2024-09-15T23:58:12.150571+0000","last_fullsized":"2024-09-15T23:58:12.150571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:48.513653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149324+0000","last_change":"2024-09-15T23:58:11.162524+0000","last_active":"2024-09-15T23:58:12.149324+0000","last_peered":"2024-09-15T23:58:12.149324+0000","last_clean":"2024-09-15T23:58:12.149324+0000","last_became_active":"2024-09-15T23:58:11.161994+0000","last_became_peered":"2024-09-15T23:58:11.161994+0000","last_unstale":"2024-09-15T23:58:12.149324+0000","last_undegraded":"2024-09-15T23:58:12.149324+0000","last_fullsized":"2024-09-15T23:58:12.149324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:25.333515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150250+0000","last_change":"2024-09-15T23:58:11.143229+0000","last_active":"2024-09-15T23:58:12.150250+0000","last_peered":"2024-09-15T23:58:12.150250+0000","last_clean":"2024-09-15T23:58:12.150250+0000","last_became_active":"2024-09-15T23:58:11.142346+0000","last_became_peered":"2024-09-15T23:58:11.142346+0000","last_unstale":"2024-09-15T23:58:12.150250+0000","last_undegraded":"2024-09-15T23:58:12.150250+0000","last_fullsized":"2024-09-15T23:58:12.150250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:51.414916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136944+0000","last_change":"2024-09-15T23:58:11.152092+0000","last_active":"2024-09-15T23:58:12.136944+0000","last_peered":"2024-09-15T23:58:12.136944+0000","last_clean":"2024-09-15T23:58:12.136944+0000","last_became_active":"2024-09-15T23:58:11.151769+0000","last_became_peered":"2024-09-15T23:58:11.151769+0000","last_unstale":"2024-09-15T23:58:12.136944+0000","last_undegraded":"2024-09-15T23:58:12.136944+0000","last_fullsized":"2024-09-15T23:58:12.136944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:42.690545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150191+0000","last_change":"2024-09-15T23:58:11.158671+0000","last_active":"2024-09-15T23:58:12.150191+0000","last_peered":"2024-09-15T23:58:12.150191+0000","last_clean":"2024-09-15T23:58:12.150191+0000","last_became_active":"2024-09-15T23:58:11.158347+0000","last_became_peered":"2024-09-15T23:58:11.158347+0000","last_unstale":"2024-09-15T23:58:12.150191+0000","last_undegraded":"2024-09-15T23:58:12.150191+0000","last_fullsized":"2024-09-15T23:58:12.150191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:52.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136298+0000","last_change":"2024-09-15T23:58:11.149725+0000","last_active":"2024-09-15T23:58:12.136298+0000","last_peered":"2024-09-15T23:58:12.136298+0000","last_clean":"2024-09-15T23:58:12.136298+0000","last_became_active":"2024-09-15T23:58:11.149388+0000","last_became_peered":"2024-09-15T23:58:11.149388+0000","last_unstale":"2024-09-15T23:58:12.136298+0000","last_undegraded":"2024-09-15T23:58:12.136298+0000","last_fullsized":"2024-09-15T23:58:12.136298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:54.807443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149128+0000","last_change":"2024-09-15T23:58:11.155976+0000","last_active":"2024-09-15T23:58:12.149128+0000","last_peered":"2024-09-15T23:58:12.149128+0000","last_clean":"2024-09-15T23:58:12.149128+0000","last_became_active":"2024-09-15T23:58:11.155659+0000","last_became_peered":"2024-09-15T23:58:11.155659+0000","last_unstale":"2024-09-15T23:58:12.149128+0000","last_undegraded":"2024-09-15T23:58:12.149128+0000","last_fullsized":"2024-09-15T23:58:12.149128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:21.524908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137172+0000","last_change":"2024-09-15T23:58:11.157326+0000","last_active":"2024-09-15T23:58:12.137172+0000","last_peered":"2024-09-15T23:58:12.137172+0000","last_clean":"2024-09-15T23:58:12.137172+0000","last_became_active":"2024-09-15T23:58:11.156944+0000","last_became_peered":"2024-09-15T23:58:11.156944+0000","last_unstale":"2024-09-15T23:58:12.137172+0000","last_undegraded":"2024-09-15T23:58:12.137172+0000","last_fullsized":"2024-09-15T23:58:12.137172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:49.913437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151176+0000","last_change":"2024-09-15T23:58:11.154418+0000","last_active":"2024-09-15T23:58:12.151176+0000","last_peered":"2024-09-15T23:58:12.151176+0000","last_clean":"2024-09-15T23:58:12.151176+0000","last_became_active":"2024-09-15T23:58:11.154081+0000","last_became_peered":"2024-09-15T23:58:11.154081+0000","last_unstale":"2024-09-15T23:58:12.151176+0000","last_undegraded":"2024-09-15T23:58:12.151176+0000","last_fullsized":"2024-09-15T23:58:12.151176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:09.920736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149482+0000","last_change":"2024-09-15T23:58:11.158224+0000","last_active":"2024-09-15T23:58:12.149482+0000","last_peered":"2024-09-15T23:58:12.149482+0000","last_clean":"2024-09-15T23:58:12.149482+0000","last_became_active":"2024-09-15T23:58:11.156076+0000","last_became_peered":"2024-09-15T23:58:11.156076+0000","last_unstale":"2024-09-15T23:58:12.149482+0000","last_undegraded":"2024-09-15T23:58:12.149482+0000","last_fullsized":"2024-09-15T23:58:12.149482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:29.448669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148503+0000","last_change":"2024-09-15T23:58:11.151904+0000","last_active":"2024-09-15T23:58:12.148503+0000","last_peered":"2024-09-15T23:58:12.148503+0000","last_clean":"2024-09-15T23:58:12.148503+0000","last_became_active":"2024-09-15T23:58:11.151564+0000","last_became_peered":"2024-09-15T23:58:11.151564+0000","last_unstale":"2024-09-15T23:58:12.148503+0000","last_undegraded":"2024-09-15T23:58:12.148503+0000","last_fullsized":"2024-09-15T23:58:12.148503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:11.297350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151021+0000","last_change":"2024-09-15T23:58:11.148139+0000","last_active":"2024-09-15T23:58:12.151021+0000","last_peered":"2024-09-15T23:58:12.151021+0000","last_clean":"2024-09-15T23:58:12.151021+0000","last_became_active":"2024-09-15T23:58:11.147820+0000","last_became_peered":"2024-09-15T23:58:11.147820+0000","last_unstale":"2024-09-15T23:58:12.151021+0000","last_undegraded":"2024-09-15T23:58:12.151021+0000","last_fullsized":"2024-09-15T23:58:12.151021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:33.786315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150504+0000","last_change":"2024-09-15T23:58:11.160284+0000","last_active":"2024-09-15T23:58:12.150504+0000","last_peered":"2024-09-15T23:58:12.150504+0000","last_clean":"2024-09-15T23:58:12.150504+0000","last_became_active":"2024-09-15T23:58:11.159931+0000","last_became_peered":"2024-09-15T23:58:11.159931+0000","last_unstale":"2024-09-15T23:58:12.150504+0000","last_undegraded":"2024-09-15T23:58:12.150504+0000","last_fullsized":"2024-09-15T23:58:12.150504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:35.381030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147613+0000","last_change":"2024-09-15T23:58:11.162637+0000","last_active":"2024-09-15T23:58:12.147613+0000","last_peered":"2024-09-15T23:58:12.147613+0000","last_clean":"2024-09-15T23:58:12.147613+0000","last_became_active":"2024-09-15T23:58:11.162322+0000","last_became_peered":"2024-09-15T23:58:11.162322+0000","last_unstale":"2024-09-15T23:58:12.147613+0000","last_undegraded":"2024-09-15T23:58:12.147613+0000","last_fullsized":"2024-09-15T23:58:12.147613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:50.489012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154813+0000","last_change":"2024-09-15T23:58:11.160068+0000","last_active":"2024-09-15T23:58:12.154813+0000","last_peered":"2024-09-15T23:58:12.154813+0000","last_clean":"2024-09-15T23:58:12.154813+0000","last_became_active":"2024-09-15T23:58:11.159553+0000","last_became_peered":"2024-09-15T23:58:11.159553+0000","last_unstale":"2024-09-15T23:58:12.154813+0000","last_undegraded":"2024-09-15T23:58:12.154813+0000","last_fullsized":"2024-09-15T23:58:12.154813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:31.962049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147516+0000","last_change":"2024-09-15T23:58:11.148755+0000","last_active":"2024-09-15T23:58:12.147516+0000","last_peered":"2024-09-15T23:58:12.147516+0000","last_clean":"2024-09-15T23:58:12.147516+0000","last_became_active":"2024-09-15T23:58:11.148428+0000","last_became_peered":"2024-09-15T23:58:11.148428+0000","last_unstale":"2024-09-15T23:58:12.147516+0000","last_undegraded":"2024-09-15T23:58:12.147516+0000","last_fullsized":"2024-09-15T23:58:12.147516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.119847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148320+0000","last_change":"2024-09-15T23:58:11.143374+0000","last_active":"2024-09-15T23:58:12.148320+0000","last_peered":"2024-09-15T23:58:12.148320+0000","last_clean":"2024-09-15T23:58:12.148320+0000","last_became_active":"2024-09-15T23:58:11.143032+0000","last_became_peered":"2024-09-15T23:58:11.143032+0000","last_unstale":"2024-09-15T23:58:12.148320+0000","last_undegraded":"2024-09-15T23:58:12.148320+0000","last_fullsized":"2024-09-15T23:58:12.148320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:29.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153498+0000","last_change":"2024-09-15T23:58:11.160524+0000","last_active":"2024-09-15T23:58:12.153498+0000","last_peered":"2024-09-15T23:58:12.153498+0000","last_clean":"2024-09-15T23:58:12.153498+0000","last_became_active":"2024-09-15T23:58:11.160141+0000","last_became_peered":"2024-09-15T23:58:11.160141+0000","last_unstale":"2024-09-15T23:58:12.153498+0000","last_undegraded":"2024-09-15T23:58:12.153498+0000","last_fullsized":"2024-09-15T23:58:12.153498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:36.730764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148654+0000","last_change":"2024-09-15T23:58:11.153493+0000","last_active":"2024-09-15T23:58:12.148654+0000","last_peered":"2024-09-15T23:58:12.148654+0000","last_clean":"2024-09-15T23:58:12.148654+0000","last_became_active":"2024-09-15T23:58:11.153026+0000","last_became_peered":"2024-09-15T23:58:11.153026+0000","last_unstale":"2024-09-15T23:58:12.148654+0000","last_undegraded":"2024-09-15T23:58:12.148654+0000","last_fullsized":"2024-09-15T23:58:12.148654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:15.700186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148212+0000","last_change":"2024-09-15T23:58:11.159991+0000","last_active":"2024-09-15T23:58:12.148212+0000","last_peered":"2024-09-15T23:58:12.148212+0000","last_clean":"2024-09-15T23:58:12.148212+0000","last_became_active":"2024-09-15T23:58:11.159656+0000","last_became_peered":"2024-09-15T23:58:11.159656+0000","last_unstale":"2024-09-15T23:58:12.148212+0000","last_undegraded":"2024-09-15T23:58:12.148212+0000","last_fullsized":"2024-09-15T23:58:12.148212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:53.065297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136354+0000","last_change":"2024-09-15T23:58:11.154058+0000","last_active":"2024-09-15T23:58:12.136354+0000","last_peered":"2024-09-15T23:58:12.136354+0000","last_clean":"2024-09-15T23:58:12.136354+0000","last_became_active":"2024-09-15T23:58:11.153769+0000","last_became_peered":"2024-09-15T23:58:11.153769+0000","last_unstale":"2024-09-15T23:58:12.136354+0000","last_undegraded":"2024-09-15T23:58:12.136354+0000","last_fullsized":"2024-09-15T23:58:12.136354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:03:54.768080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136887+0000","last_change":"2024-09-15T23:58:11.148173+0000","last_active":"2024-09-15T23:58:12.136887+0000","last_peered":"2024-09-15T23:58:12.136887+0000","last_clean":"2024-09-15T23:58:12.136887+0000","last_became_active":"2024-09-15T23:58:11.147843+0000","last_became_peered":"2024-09-15T23:58:11.147843+0000","last_unstale":"2024-09-15T23:58:12.136887+0000","last_undegraded":"2024-09-15T23:58:12.136887+0000","last_fullsized":"2024-09-15T23:58:12.136887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:29.789888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136563+0000","last_change":"2024-09-15T23:58:11.162113+0000","last_active":"2024-09-15T23:58:12.136563+0000","last_peered":"2024-09-15T23:58:12.136563+0000","last_clean":"2024-09-15T23:58:12.136563+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.136563+0000","last_undegraded":"2024-09-15T23:58:12.136563+0000","last_fullsized":"2024-09-15T23:58:12.136563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:05.334727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148712+0000","last_change":"2024-09-15T23:58:11.150172+0000","last_active":"2024-09-15T23:58:12.148712+0000","last_peered":"2024-09-15T23:58:12.148712+0000","last_clean":"2024-09-15T23:58:12.148712+0000","last_became_active":"2024-09-15T23:58:11.149800+0000","last_became_peered":"2024-09-15T23:58:11.149800+0000","last_unstale":"2024-09-15T23:58:12.148712+0000","last_undegraded":"2024-09-15T23:58:12.148712+0000","last_fullsized":"2024-09-15T23:58:12.148712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:16.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154255+0000","last_change":"2024-09-15T23:58:11.153829+0000","last_active":"2024-09-15T23:58:12.154255+0000","last_peered":"2024-09-15T23:58:12.154255+0000","last_clean":"2024-09-15T23:58:12.154255+0000","last_became_active":"2024-09-15T23:58:11.153512+0000","last_became_peered":"2024-09-15T23:58:11.153512+0000","last_unstale":"2024-09-15T23:58:12.154255+0000","last_undegraded":"2024-09-15T23:58:12.154255+0000","last_fullsized":"2024-09-15T23:58:12.154255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:36.405204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147909+0000","last_change":"2024-09-15T23:58:11.162122+0000","last_active":"2024-09-15T23:58:12.147909+0000","last_peered":"2024-09-15T23:58:12.147909+0000","last_clean":"2024-09-15T23:58:12.147909+0000","last_became_active":"2024-09-15T23:58:11.161814+0000","last_became_peered":"2024-09-15T23:58:11.161814+0000","last_unstale":"2024-09-15T23:58:12.147909+0000","last_undegraded":"2024-09-15T23:58:12.147909+0000","last_fullsized":"2024-09-15T23:58:12.147909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:07.058458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150959+0000","last_change":"2024-09-15T23:58:11.157204+0000","last_active":"2024-09-15T23:58:12.150959+0000","last_peered":"2024-09-15T23:58:12.150959+0000","last_clean":"2024-09-15T23:58:12.150959+0000","last_became_active":"2024-09-15T23:58:11.156906+0000","last_became_peered":"2024-09-15T23:58:11.156906+0000","last_unstale":"2024-09-15T23:58:12.150959+0000","last_undegraded":"2024-09-15T23:58:12.150959+0000","last_fullsized":"2024-09-15T23:58:12.150959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:04.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148236+0000","last_change":"2024-09-15T23:58:11.150191+0000","last_active":"2024-09-15T23:58:12.148236+0000","last_peered":"2024-09-15T23:58:12.148236+0000","last_clean":"2024-09-15T23:58:12.148236+0000","last_became_active":"2024-09-15T23:58:11.149616+0000","last_became_peered":"2024-09-15T23:58:11.149616+0000","last_unstale":"2024-09-15T23:58:12.148236+0000","last_undegraded":"2024-09-15T23:58:12.148236+0000","last_fullsized":"2024-09-15T23:58:12.148236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:49.956626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148515+0000","last_change":"2024-09-15T23:58:11.162127+0000","last_active":"2024-09-15T23:58:12.148515+0000","last_peered":"2024-09-15T23:58:12.148515+0000","last_clean":"2024-09-15T23:58:12.148515+0000","last_became_active":"2024-09-15T23:58:11.161766+0000","last_became_peered":"2024-09-15T23:58:11.161766+0000","last_unstale":"2024-09-15T23:58:12.148515+0000","last_undegraded":"2024-09-15T23:58:12.148515+0000","last_fullsized":"2024-09-15T23:58:12.148515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:09.626747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154236+0000","last_change":"2024-09-15T23:58:11.153546+0000","last_active":"2024-09-15T23:58:12.154236+0000","last_peered":"2024-09-15T23:58:12.154236+0000","last_clean":"2024-09-15T23:58:12.154236+0000","last_became_active":"2024-09-15T23:58:11.153116+0000","last_became_peered":"2024-09-15T23:58:11.153116+0000","last_unstale":"2024-09-15T23:58:12.154236+0000","last_undegraded":"2024-09-15T23:58:12.154236+0000","last_fullsized":"2024-09-15T23:58:12.154236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:02.389068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148125+0000","last_change":"2024-09-15T23:58:11.147446+0000","last_active":"2024-09-15T23:58:12.148125+0000","last_peered":"2024-09-15T23:58:12.148125+0000","last_clean":"2024-09-15T23:58:12.148125+0000","last_became_active":"2024-09-15T23:58:11.146285+0000","last_became_peered":"2024-09-15T23:58:11.146285+0000","last_unstale":"2024-09-15T23:58:12.148125+0000","last_undegraded":"2024-09-15T23:58:12.148125+0000","last_fullsized":"2024-09-15T23:58:12.148125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:38.741814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":76,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:13.002001+0000","last_change":"2024-09-15T23:58:09.123719+0000","last_active":"2024-09-15T23:59:13.002001+0000","last_peered":"2024-09-15T23:59:13.002001+0000","last_clean":"2024-09-15T23:59:13.002001+0000","last_became_active":"2024-09-15T23:58:09.123082+0000","last_became_peered":"2024-09-15T23:58:09.123082+0000","last_unstale":"2024-09-15T23:59:13.002001+0000","last_undegraded":"2024-09-15T23:59:13.002001+0000","last_fullsized":"2024-09-15T23:59:13.002001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:46.704121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137195+0000","last_change":"2024-09-15T23:58:11.162571+0000","last_active":"2024-09-15T23:58:12.137195+0000","last_peered":"2024-09-15T23:58:12.137195+0000","last_clean":"2024-09-15T23:58:12.137195+0000","last_became_active":"2024-09-15T23:58:11.161784+0000","last_became_peered":"2024-09-15T23:58:11.161784+0000","last_unstale":"2024-09-15T23:58:12.137195+0000","last_undegraded":"2024-09-15T23:58:12.137195+0000","last_fullsized":"2024-09-15T23:58:12.137195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:58.134589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.290419+0000","last_change":"2024-09-15T23:58:09.125683+0000","last_active":"2024-09-15T23:58:22.290419+0000","last_peered":"2024-09-15T23:58:22.290419+0000","last_clean":"2024-09-15T23:58:22.290419+0000","last_became_active":"2024-09-15T23:58:09.125345+0000","last_became_peered":"2024-09-15T23:58:09.125345+0000","last_unstale":"2024-09-15T23:58:22.290419+0000","last_undegraded":"2024-09-15T23:58:22.290419+0000","last_fullsized":"2024-09-15T23:58:22.290419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:39.998850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284754+0000","last_change":"2024-09-15T23:58:09.116886+0000","last_active":"2024-09-15T23:58:21.284754+0000","last_peered":"2024-09-15T23:58:21.284754+0000","last_clean":"2024-09-15T23:58:21.284754+0000","last_became_active":"2024-09-15T23:58:09.116510+0000","last_became_peered":"2024-09-15T23:58:09.116510+0000","last_unstale":"2024-09-15T23:58:21.284754+0000","last_undegraded":"2024-09-15T23:58:21.284754+0000","last_fullsized":"2024-09-15T23:58:21.284754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:10.368106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136363+0000","last_change":"2024-09-15T23:58:11.144759+0000","last_active":"2024-09-15T23:58:12.136363+0000","last_peered":"2024-09-15T23:58:12.136363+0000","last_clean":"2024-09-15T23:58:12.136363+0000","last_became_active":"2024-09-15T23:58:11.144435+0000","last_became_peered":"2024-09-15T23:58:11.144435+0000","last_unstale":"2024-09-15T23:58:12.136363+0000","last_undegraded":"2024-09-15T23:58:12.136363+0000","last_fullsized":"2024-09-15T23:58:12.136363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:32.787727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300459+0000","last_change":"2024-09-15T23:58:09.124938+0000","last_active":"2024-09-15T23:58:23.300459+0000","last_peered":"2024-09-15T23:58:23.300459+0000","last_clean":"2024-09-15T23:58:23.300459+0000","last_became_active":"2024-09-15T23:58:09.124532+0000","last_became_peered":"2024-09-15T23:58:09.124532+0000","last_unstale":"2024-09-15T23:58:23.300459+0000","last_undegraded":"2024-09-15T23:58:23.300459+0000","last_fullsized":"2024-09-15T23:58:23.300459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148543+0000","last_change":"2024-09-15T23:58:11.145940+0000","last_active":"2024-09-15T23:58:12.148543+0000","last_peered":"2024-09-15T23:58:12.148543+0000","last_clean":"2024-09-15T23:58:12.148543+0000","last_became_active":"2024-09-15T23:58:11.145642+0000","last_became_peered":"2024-09-15T23:58:11.145642+0000","last_unstale":"2024-09-15T23:58:12.148543+0000","last_undegraded":"2024-09-15T23:58:12.148543+0000","last_fullsized":"2024-09-15T23:58:12.148543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:58.241337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:17.006890+0000","last_change":"2024-09-15T23:58:09.126742+0000","last_active":"2024-09-15T23:59:17.006890+0000","last_peered":"2024-09-15T23:59:17.006890+0000","last_clean":"2024-09-15T23:59:17.006890+0000","last_became_active":"2024-09-15T23:58:09.126432+0000","last_became_peered":"2024-09-15T23:58:09.126432+0000","last_unstale":"2024-09-15T23:59:17.006890+0000","last_undegraded":"2024-09-15T23:59:17.006890+0000","last_fullsized":"2024-09-15T23:59:17.006890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:14:11.582346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148667+0000","last_change":"2024-09-15T23:58:11.150130+0000","last_active":"2024-09-15T23:58:12.148667+0000","last_peered":"2024-09-15T23:58:12.148667+0000","last_clean":"2024-09-15T23:58:12.148667+0000","last_became_active":"2024-09-15T23:58:11.149531+0000","last_became_peered":"2024-09-15T23:58:11.149531+0000","last_unstale":"2024-09-15T23:58:12.148667+0000","last_undegraded":"2024-09-15T23:58:12.148667+0000","last_fullsized":"2024-09-15T23:58:12.148667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:07.206505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299397+0000","last_change":"2024-09-15T23:58:09.121229+0000","last_active":"2024-09-15T23:58:23.299397+0000","last_peered":"2024-09-15T23:58:23.299397+0000","last_clean":"2024-09-15T23:58:23.299397+0000","last_became_active":"2024-09-15T23:58:09.120808+0000","last_became_peered":"2024-09-15T23:58:09.120808+0000","last_unstale":"2024-09-15T23:58:23.299397+0000","last_undegraded":"2024-09-15T23:58:23.299397+0000","last_fullsized":"2024-09-15T23:58:23.299397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:34.155774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151252+0000","last_change":"2024-09-15T23:58:11.157253+0000","last_active":"2024-09-15T23:58:12.151252+0000","last_peered":"2024-09-15T23:58:12.151252+0000","last_clean":"2024-09-15T23:58:12.151252+0000","last_became_active":"2024-09-15T23:58:11.156952+0000","last_became_peered":"2024-09-15T23:58:11.156952+0000","last_unstale":"2024-09-15T23:58:12.151252+0000","last_undegraded":"2024-09-15T23:58:12.151252+0000","last_fullsized":"2024-09-15T23:58:12.151252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:09.840371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289729+0000","last_change":"2024-09-15T23:58:09.322776+0000","last_active":"2024-09-15T23:58:22.289729+0000","last_peered":"2024-09-15T23:58:22.289729+0000","last_clean":"2024-09-15T23:58:22.289729+0000","last_became_active":"2024-09-15T23:58:09.322304+0000","last_became_peered":"2024-09-15T23:58:09.322304+0000","last_unstale":"2024-09-15T23:58:22.289729+0000","last_undegraded":"2024-09-15T23:58:22.289729+0000","last_fullsized":"2024-09-15T23:58:22.289729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:51.413578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148251+0000","last_change":"2024-09-15T23:58:11.165067+0000","last_active":"2024-09-15T23:58:12.148251+0000","last_peered":"2024-09-15T23:58:12.148251+0000","last_clean":"2024-09-15T23:58:12.148251+0000","last_became_active":"2024-09-15T23:58:11.163722+0000","last_became_peered":"2024-09-15T23:58:11.163722+0000","last_unstale":"2024-09-15T23:58:12.148251+0000","last_undegraded":"2024-09-15T23:58:12.148251+0000","last_fullsized":"2024-09-15T23:58:12.148251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:01.354822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":165,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:17.006630+0000","last_change":"2024-09-15T23:58:09.128077+0000","last_active":"2024-09-15T23:59:17.006630+0000","last_peered":"2024-09-15T23:59:17.006630+0000","last_clean":"2024-09-15T23:59:17.006630+0000","last_became_active":"2024-09-15T23:58:09.127215+0000","last_became_peered":"2024-09-15T23:58:09.127215+0000","last_unstale":"2024-09-15T23:59:17.006630+0000","last_undegraded":"2024-09-15T23:59:17.006630+0000","last_fullsized":"2024-09-15T23:59:17.006630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:47.875279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":118,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150490+0000","last_change":"2024-09-15T23:58:11.156020+0000","last_active":"2024-09-15T23:58:12.150490+0000","last_peered":"2024-09-15T23:58:12.150490+0000","last_clean":"2024-09-15T23:58:12.150490+0000","last_became_active":"2024-09-15T23:58:11.155634+0000","last_became_peered":"2024-09-15T23:58:11.155634+0000","last_unstale":"2024-09-15T23:58:12.150490+0000","last_undegraded":"2024-09-15T23:58:12.150490+0000","last_fullsized":"2024-09-15T23:58:12.150490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934453+0000","last_change":"2024-09-15T23:58:09.123084+0000","last_active":"2024-09-15T23:58:23.934453+0000","last_peered":"2024-09-15T23:58:23.934453+0000","last_clean":"2024-09-15T23:58:23.934453+0000","last_became_active":"2024-09-15T23:58:09.122594+0000","last_became_peered":"2024-09-15T23:58:09.122594+0000","last_unstale":"2024-09-15T23:58:23.934453+0000","last_undegraded":"2024-09-15T23:58:23.934453+0000","last_fullsized":"2024-09-15T23:58:23.934453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:47.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.149295+0000","last_change":"2024-09-15T23:58:11.155082+0000","last_active":"2024-09-15T23:58:12.149295+0000","last_peered":"2024-09-15T23:58:12.149295+0000","last_clean":"2024-09-15T23:58:12.149295+0000","last_became_active":"2024-09-15T23:58:11.154764+0000","last_became_peered":"2024-09-15T23:58:11.154764+0000","last_unstale":"2024-09-15T23:58:12.149295+0000","last_undegraded":"2024-09-15T23:58:12.149295+0000","last_fullsized":"2024-09-15T23:58:12.149295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:47.170383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300267+0000","last_change":"2024-09-15T23:58:09.124416+0000","last_active":"2024-09-15T23:58:23.300267+0000","last_peered":"2024-09-15T23:58:23.300267+0000","last_clean":"2024-09-15T23:58:23.300267+0000","last_became_active":"2024-09-15T23:58:09.123836+0000","last_became_peered":"2024-09-15T23:58:09.123836+0000","last_unstale":"2024-09-15T23:58:23.300267+0000","last_undegraded":"2024-09-15T23:58:23.300267+0000","last_fullsized":"2024-09-15T23:58:23.300267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:35.730677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150934+0000","last_change":"2024-09-15T23:58:11.162220+0000","last_active":"2024-09-15T23:58:12.150934+0000","last_peered":"2024-09-15T23:58:12.150934+0000","last_clean":"2024-09-15T23:58:12.150934+0000","last_became_active":"2024-09-15T23:58:11.161887+0000","last_became_peered":"2024-09-15T23:58:11.161887+0000","last_unstale":"2024-09-15T23:58:12.150934+0000","last_undegraded":"2024-09-15T23:58:12.150934+0000","last_fullsized":"2024-09-15T23:58:12.150934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:35.070135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:14.002832+0000","last_change":"2024-09-15T23:58:09.123010+0000","last_active":"2024-09-15T23:59:14.002832+0000","last_peered":"2024-09-15T23:59:14.002832+0000","last_clean":"2024-09-15T23:59:14.002832+0000","last_became_active":"2024-09-15T23:58:09.122615+0000","last_became_peered":"2024-09-15T23:58:09.122615+0000","last_unstale":"2024-09-15T23:59:14.002832+0000","last_undegraded":"2024-09-15T23:59:14.002832+0000","last_fullsized":"2024-09-15T23:59:14.002832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:09.634409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150261+0000","last_change":"2024-09-15T23:58:11.159069+0000","last_active":"2024-09-15T23:58:12.150261+0000","last_peered":"2024-09-15T23:58:12.150261+0000","last_clean":"2024-09-15T23:58:12.150261+0000","last_became_active":"2024-09-15T23:58:11.158659+0000","last_became_peered":"2024-09-15T23:58:11.158659+0000","last_unstale":"2024-09-15T23:58:12.150261+0000","last_undegraded":"2024-09-15T23:58:12.150261+0000","last_fullsized":"2024-09-15T23:58:12.150261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:24.336089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298401+0000","last_change":"2024-09-15T23:58:09.322903+0000","last_active":"2024-09-15T23:58:23.298401+0000","last_peered":"2024-09-15T23:58:23.298401+0000","last_clean":"2024-09-15T23:58:23.298401+0000","last_became_active":"2024-09-15T23:58:09.322326+0000","last_became_peered":"2024-09-15T23:58:09.322326+0000","last_unstale":"2024-09-15T23:58:23.298401+0000","last_undegraded":"2024-09-15T23:58:23.298401+0000","last_fullsized":"2024-09-15T23:58:23.298401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:58.749293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136919+0000","last_change":"2024-09-15T23:58:11.162104+0000","last_active":"2024-09-15T23:58:12.136919+0000","last_peered":"2024-09-15T23:58:12.136919+0000","last_clean":"2024-09-15T23:58:12.136919+0000","last_became_active":"2024-09-15T23:58:11.161739+0000","last_became_peered":"2024-09-15T23:58:11.161739+0000","last_unstale":"2024-09-15T23:58:12.136919+0000","last_undegraded":"2024-09-15T23:58:12.136919+0000","last_fullsized":"2024-09-15T23:58:12.136919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:55.316245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.279815+0000","last_change":"2024-09-15T23:58:09.119975+0000","last_active":"2024-09-15T23:58:22.279815+0000","last_peered":"2024-09-15T23:58:22.279815+0000","last_clean":"2024-09-15T23:58:22.279815+0000","last_became_active":"2024-09-15T23:58:09.119340+0000","last_became_peered":"2024-09-15T23:58:09.119340+0000","last_unstale":"2024-09-15T23:58:22.279815+0000","last_undegraded":"2024-09-15T23:58:22.279815+0000","last_fullsized":"2024-09-15T23:58:22.279815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:22:14.858076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153874+0000","last_change":"2024-09-15T23:58:11.157389+0000","last_active":"2024-09-15T23:58:12.153874+0000","last_peered":"2024-09-15T23:58:12.153874+0000","last_clean":"2024-09-15T23:58:12.153874+0000","last_became_active":"2024-09-15T23:58:11.157033+0000","last_became_peered":"2024-09-15T23:58:11.157033+0000","last_unstale":"2024-09-15T23:58:12.153874+0000","last_undegraded":"2024-09-15T23:58:12.153874+0000","last_fullsized":"2024-09-15T23:58:12.153874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:49.149860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.299541+0000","last_change":"2024-09-15T23:58:09.120937+0000","last_active":"2024-09-15T23:58:23.299541+0000","last_peered":"2024-09-15T23:58:23.299541+0000","last_clean":"2024-09-15T23:58:23.299541+0000","last_became_active":"2024-09-15T23:58:09.120586+0000","last_became_peered":"2024-09-15T23:58:09.120586+0000","last_unstale":"2024-09-15T23:58:23.299541+0000","last_undegraded":"2024-09-15T23:58:23.299541+0000","last_fullsized":"2024-09-15T23:58:23.299541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:18.432631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150943+0000","last_change":"2024-09-15T23:58:11.155316+0000","last_active":"2024-09-15T23:58:12.150943+0000","last_peered":"2024-09-15T23:58:12.150943+0000","last_clean":"2024-09-15T23:58:12.150943+0000","last_became_active":"2024-09-15T23:58:11.154975+0000","last_became_peered":"2024-09-15T23:58:11.154975+0000","last_unstale":"2024-09-15T23:58:12.150943+0000","last_undegraded":"2024-09-15T23:58:12.150943+0000","last_fullsized":"2024-09-15T23:58:12.150943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:09.856126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300569+0000","last_change":"2024-09-15T23:58:09.125045+0000","last_active":"2024-09-15T23:58:23.300569+0000","last_peered":"2024-09-15T23:58:23.300569+0000","last_clean":"2024-09-15T23:58:23.300569+0000","last_became_active":"2024-09-15T23:58:09.124577+0000","last_became_peered":"2024-09-15T23:58:09.124577+0000","last_unstale":"2024-09-15T23:58:23.300569+0000","last_undegraded":"2024-09-15T23:58:23.300569+0000","last_fullsized":"2024-09-15T23:58:23.300569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:48.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150843+0000","last_change":"2024-09-15T23:58:11.160241+0000","last_active":"2024-09-15T23:58:12.150843+0000","last_peered":"2024-09-15T23:58:12.150843+0000","last_clean":"2024-09-15T23:58:12.150843+0000","last_became_active":"2024-09-15T23:58:11.159886+0000","last_became_peered":"2024-09-15T23:58:11.159886+0000","last_unstale":"2024-09-15T23:58:12.150843+0000","last_undegraded":"2024-09-15T23:58:12.150843+0000","last_fullsized":"2024-09-15T23:58:12.150843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:02.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:16.016352+0000","last_change":"2024-09-15T23:58:09.482888+0000","last_active":"2024-09-15T23:59:16.016352+0000","last_peered":"2024-09-15T23:59:16.016352+0000","last_clean":"2024-09-15T23:59:16.016352+0000","last_became_active":"2024-09-15T23:58:09.482497+0000","last_became_peered":"2024-09-15T23:58:09.482497+0000","last_unstale":"2024-09-15T23:59:16.016352+0000","last_undegraded":"2024-09-15T23:59:16.016352+0000","last_fullsized":"2024-09-15T23:59:16.016352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:59.320794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151266+0000","last_change":"2024-09-15T23:58:11.159452+0000","last_active":"2024-09-15T23:58:12.151266+0000","last_peered":"2024-09-15T23:58:12.151266+0000","last_clean":"2024-09-15T23:58:12.151266+0000","last_became_active":"2024-09-15T23:58:11.157610+0000","last_became_peered":"2024-09-15T23:58:11.157610+0000","last_unstale":"2024-09-15T23:58:12.151266+0000","last_undegraded":"2024-09-15T23:58:12.151266+0000","last_fullsized":"2024-09-15T23:58:12.151266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:17.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150373+0000","last_change":"2024-09-15T23:56:56.876989+0000","last_active":"2024-09-15T23:58:12.150373+0000","last_peered":"2024-09-15T23:58:12.150373+0000","last_clean":"2024-09-15T23:58:12.150373+0000","last_became_active":"2024-09-15T23:56:56.876413+0000","last_became_peered":"2024-09-15T23:56:56.876413+0000","last_unstale":"2024-09-15T23:58:12.150373+0000","last_undegraded":"2024-09-15T23:58:12.150373+0000","last_fullsized":"2024-09-15T23:58:12.150373+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:53:27.674229+0000","last_clean_scrub_stamp":"2024-09-15T23:53:27.674229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:06.674196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934257+0000","last_change":"2024-09-15T23:58:09.124514+0000","last_active":"2024-09-15T23:58:23.934257+0000","last_peered":"2024-09-15T23:58:23.934257+0000","last_clean":"2024-09-15T23:58:23.934257+0000","last_became_active":"2024-09-15T23:58:09.122124+0000","last_became_peered":"2024-09-15T23:58:09.122124+0000","last_unstale":"2024-09-15T23:58:23.934257+0000","last_undegraded":"2024-09-15T23:58:23.934257+0000","last_fullsized":"2024-09-15T23:58:23.934257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:13.563073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136568+0000","last_change":"2024-09-15T23:58:11.149559+0000","last_active":"2024-09-15T23:58:12.136568+0000","last_peered":"2024-09-15T23:58:12.136568+0000","last_clean":"2024-09-15T23:58:12.136568+0000","last_became_active":"2024-09-15T23:58:11.149215+0000","last_became_peered":"2024-09-15T23:58:11.149215+0000","last_unstale":"2024-09-15T23:58:12.136568+0000","last_undegraded":"2024-09-15T23:58:12.136568+0000","last_fullsized":"2024-09-15T23:58:12.136568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:16.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:21.284817+0000","last_change":"2024-09-15T23:58:09.127770+0000","last_active":"2024-09-15T23:58:21.284817+0000","last_peered":"2024-09-15T23:58:21.284817+0000","last_clean":"2024-09-15T23:58:21.284817+0000","last_became_active":"2024-09-15T23:58:09.127404+0000","last_became_peered":"2024-09-15T23:58:09.127404+0000","last_unstale":"2024-09-15T23:58:21.284817+0000","last_undegraded":"2024-09-15T23:58:21.284817+0000","last_fullsized":"2024-09-15T23:58:21.284817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:48.176290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148872+0000","last_change":"2024-09-15T23:58:11.162161+0000","last_active":"2024-09-15T23:58:12.148872+0000","last_peered":"2024-09-15T23:58:12.148872+0000","last_clean":"2024-09-15T23:58:12.148872+0000","last_became_active":"2024-09-15T23:58:11.161879+0000","last_became_peered":"2024-09-15T23:58:11.161879+0000","last_unstale":"2024-09-15T23:58:12.148872+0000","last_undegraded":"2024-09-15T23:58:12.148872+0000","last_fullsized":"2024-09-15T23:58:12.148872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:15.003851+0000","last_change":"2024-09-15T23:58:09.322611+0000","last_active":"2024-09-15T23:59:15.003851+0000","last_peered":"2024-09-15T23:59:15.003851+0000","last_clean":"2024-09-15T23:59:15.003851+0000","last_became_active":"2024-09-15T23:58:09.322254+0000","last_became_peered":"2024-09-15T23:58:09.322254+0000","last_unstale":"2024-09-15T23:59:15.003851+0000","last_undegraded":"2024-09-15T23:59:15.003851+0000","last_fullsized":"2024-09-15T23:59:15.003851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:03.803927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137151+0000","last_change":"2024-09-15T23:58:11.152219+0000","last_active":"2024-09-15T23:58:12.137151+0000","last_peered":"2024-09-15T23:58:12.137151+0000","last_clean":"2024-09-15T23:58:12.137151+0000","last_became_active":"2024-09-15T23:58:11.151885+0000","last_became_peered":"2024-09-15T23:58:11.151885+0000","last_unstale":"2024-09-15T23:58:12.137151+0000","last_undegraded":"2024-09-15T23:58:12.137151+0000","last_fullsized":"2024-09-15T23:58:12.137151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:10.780163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":79,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:15.014278+0000","last_change":"2024-09-15T23:58:09.125152+0000","last_active":"2024-09-15T23:59:15.014278+0000","last_peered":"2024-09-15T23:59:15.014278+0000","last_clean":"2024-09-15T23:59:15.014278+0000","last_became_active":"2024-09-15T23:58:09.124754+0000","last_became_peered":"2024-09-15T23:58:09.124754+0000","last_unstale":"2024-09-15T23:59:15.014278+0000","last_undegraded":"2024-09-15T23:59:15.014278+0000","last_fullsized":"2024-09-15T23:59:15.014278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:02.094909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136921+0000","last_change":"2024-09-15T23:58:11.160257+0000","last_active":"2024-09-15T23:58:12.136921+0000","last_peered":"2024-09-15T23:58:12.136921+0000","last_clean":"2024-09-15T23:58:12.136921+0000","last_became_active":"2024-09-15T23:58:11.159868+0000","last_became_peered":"2024-09-15T23:58:11.159868+0000","last_unstale":"2024-09-15T23:58:12.136921+0000","last_undegraded":"2024-09-15T23:58:12.136921+0000","last_fullsized":"2024-09-15T23:58:12.136921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.528002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.226786+0000","last_change":"2024-09-15T23:58:09.323034+0000","last_active":"2024-09-15T23:58:12.226786+0000","last_peered":"2024-09-15T23:58:12.226786+0000","last_clean":"2024-09-15T23:58:12.226786+0000","last_became_active":"2024-09-15T23:58:09.322658+0000","last_became_peered":"2024-09-15T23:58:09.322658+0000","last_unstale":"2024-09-15T23:58:12.226786+0000","last_undegraded":"2024-09-15T23:58:12.226786+0000","last_fullsized":"2024-09-15T23:58:12.226786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:20:11.175093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.862670+0000","last_change":"2024-09-15T23:58:11.149216+0000","last_active":"2024-09-15T23:58:12.862670+0000","last_peered":"2024-09-15T23:58:12.862670+0000","last_clean":"2024-09-15T23:58:12.862670+0000","last_became_active":"2024-09-15T23:58:11.148893+0000","last_became_peered":"2024-09-15T23:58:11.148893+0000","last_unstale":"2024-09-15T23:58:12.862670+0000","last_undegraded":"2024-09-15T23:58:12.862670+0000","last_fullsized":"2024-09-15T23:58:12.862670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:07.749101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300019+0000","last_change":"2024-09-15T23:58:09.323030+0000","last_active":"2024-09-15T23:58:23.300019+0000","last_peered":"2024-09-15T23:58:23.300019+0000","last_clean":"2024-09-15T23:58:23.300019+0000","last_became_active":"2024-09-15T23:58:09.322569+0000","last_became_peered":"2024-09-15T23:58:09.322569+0000","last_unstale":"2024-09-15T23:58:23.300019+0000","last_undegraded":"2024-09-15T23:58:23.300019+0000","last_fullsized":"2024-09-15T23:58:23.300019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:52.811395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148063+0000","last_change":"2024-09-15T23:58:11.154084+0000","last_active":"2024-09-15T23:58:12.148063+0000","last_peered":"2024-09-15T23:58:12.148063+0000","last_clean":"2024-09-15T23:58:12.148063+0000","last_became_active":"2024-09-15T23:58:11.153720+0000","last_became_peered":"2024-09-15T23:58:11.153720+0000","last_unstale":"2024-09-15T23:58:12.148063+0000","last_undegraded":"2024-09-15T23:58:12.148063+0000","last_fullsized":"2024-09-15T23:58:12.148063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:07.468812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:38.149932+0000","last_change":"2024-09-15T23:58:09.116996+0000","last_active":"2024-09-15T23:58:38.149932+0000","last_peered":"2024-09-15T23:58:38.149932+0000","last_clean":"2024-09-15T23:58:38.149932+0000","last_became_active":"2024-09-15T23:58:09.116531+0000","last_became_peered":"2024-09-15T23:58:09.116531+0000","last_unstale":"2024-09-15T23:58:38.149932+0000","last_undegraded":"2024-09-15T23:58:38.149932+0000","last_fullsized":"2024-09-15T23:58:38.149932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:23.094719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136595+0000","last_change":"2024-09-15T23:58:11.155704+0000","last_active":"2024-09-15T23:58:12.136595+0000","last_peered":"2024-09-15T23:58:12.136595+0000","last_clean":"2024-09-15T23:58:12.136595+0000","last_became_active":"2024-09-15T23:58:11.155344+0000","last_became_peered":"2024-09-15T23:58:11.155344+0000","last_unstale":"2024-09-15T23:58:12.136595+0000","last_undegraded":"2024-09-15T23:58:12.136595+0000","last_fullsized":"2024-09-15T23:58:12.136595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:50.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.298948+0000","last_change":"2024-09-15T23:58:09.482837+0000","last_active":"2024-09-15T23:58:23.298948+0000","last_peered":"2024-09-15T23:58:23.298948+0000","last_clean":"2024-09-15T23:58:23.298948+0000","last_became_active":"2024-09-15T23:58:09.482444+0000","last_became_peered":"2024-09-15T23:58:09.482444+0000","last_unstale":"2024-09-15T23:58:23.298948+0000","last_undegraded":"2024-09-15T23:58:23.298948+0000","last_fullsized":"2024-09-15T23:58:23.298948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:05.962471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150939+0000","last_change":"2024-09-15T23:58:11.157249+0000","last_active":"2024-09-15T23:58:12.150939+0000","last_peered":"2024-09-15T23:58:12.150939+0000","last_clean":"2024-09-15T23:58:12.150939+0000","last_became_active":"2024-09-15T23:58:11.156871+0000","last_became_peered":"2024-09-15T23:58:11.156871+0000","last_unstale":"2024-09-15T23:58:12.150939+0000","last_undegraded":"2024-09-15T23:58:12.150939+0000","last_fullsized":"2024-09-15T23:58:12.150939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:16.986165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279567+0000","last_change":"2024-09-15T23:58:09.324320+0000","last_active":"2024-09-15T23:58:20.279567+0000","last_peered":"2024-09-15T23:58:20.279567+0000","last_clean":"2024-09-15T23:58:20.279567+0000","last_became_active":"2024-09-15T23:58:09.323996+0000","last_became_peered":"2024-09-15T23:58:09.323996+0000","last_unstale":"2024-09-15T23:58:20.279567+0000","last_undegraded":"2024-09-15T23:58:20.279567+0000","last_fullsized":"2024-09-15T23:58:20.279567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:34.456229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.147904+0000","last_change":"2024-09-15T23:58:11.146836+0000","last_active":"2024-09-15T23:58:12.147904+0000","last_peered":"2024-09-15T23:58:12.147904+0000","last_clean":"2024-09-15T23:58:12.147904+0000","last_became_active":"2024-09-15T23:58:11.146470+0000","last_became_peered":"2024-09-15T23:58:11.146470+0000","last_unstale":"2024-09-15T23:58:12.147904+0000","last_undegraded":"2024-09-15T23:58:12.147904+0000","last_fullsized":"2024-09-15T23:58:12.147904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.721954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300504+0000","last_change":"2024-09-15T23:58:09.121920+0000","last_active":"2024-09-15T23:58:23.300504+0000","last_peered":"2024-09-15T23:58:23.300504+0000","last_clean":"2024-09-15T23:58:23.300504+0000","last_became_active":"2024-09-15T23:58:09.121605+0000","last_became_peered":"2024-09-15T23:58:09.121605+0000","last_unstale":"2024-09-15T23:58:23.300504+0000","last_undegraded":"2024-09-15T23:58:23.300504+0000","last_fullsized":"2024-09-15T23:58:23.300504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:12.707690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.154389+0000","last_change":"2024-09-15T23:58:11.153676+0000","last_active":"2024-09-15T23:58:12.154389+0000","last_peered":"2024-09-15T23:58:12.154389+0000","last_clean":"2024-09-15T23:58:12.154389+0000","last_became_active":"2024-09-15T23:58:11.152866+0000","last_became_peered":"2024-09-15T23:58:11.152866+0000","last_unstale":"2024-09-15T23:58:12.154389+0000","last_undegraded":"2024-09-15T23:58:12.154389+0000","last_fullsized":"2024-09-15T23:58:12.154389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:02.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:22.289656+0000","last_change":"2024-09-15T23:58:09.121440+0000","last_active":"2024-09-15T23:58:22.289656+0000","last_peered":"2024-09-15T23:58:22.289656+0000","last_clean":"2024-09-15T23:58:22.289656+0000","last_became_active":"2024-09-15T23:58:09.121006+0000","last_became_peered":"2024-09-15T23:58:09.121006+0000","last_unstale":"2024-09-15T23:58:22.289656+0000","last_undegraded":"2024-09-15T23:58:22.289656+0000","last_fullsized":"2024-09-15T23:58:22.289656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:32.584222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136632+0000","last_change":"2024-09-15T23:58:11.155311+0000","last_active":"2024-09-15T23:58:12.136632+0000","last_peered":"2024-09-15T23:58:12.136632+0000","last_clean":"2024-09-15T23:58:12.136632+0000","last_became_active":"2024-09-15T23:58:11.154954+0000","last_became_peered":"2024-09-15T23:58:11.154954+0000","last_unstale":"2024-09-15T23:58:12.136632+0000","last_undegraded":"2024-09-15T23:58:12.136632+0000","last_fullsized":"2024-09-15T23:58:12.136632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:52.131451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:20.279891+0000","last_change":"2024-09-15T23:58:09.126143+0000","last_active":"2024-09-15T23:58:20.279891+0000","last_peered":"2024-09-15T23:58:20.279891+0000","last_clean":"2024-09-15T23:58:20.279891+0000","last_became_active":"2024-09-15T23:58:09.125785+0000","last_became_peered":"2024-09-15T23:58:09.125785+0000","last_unstale":"2024-09-15T23:58:20.279891+0000","last_undegraded":"2024-09-15T23:58:20.279891+0000","last_fullsized":"2024-09-15T23:58:20.279891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:22.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.150690+0000","last_change":"2024-09-15T23:58:11.160834+0000","last_active":"2024-09-15T23:58:12.150690+0000","last_peered":"2024-09-15T23:58:12.150690+0000","last_clean":"2024-09-15T23:58:12.150690+0000","last_became_active":"2024-09-15T23:58:11.160458+0000","last_became_peered":"2024-09-15T23:58:11.160458+0000","last_unstale":"2024-09-15T23:58:12.150690+0000","last_undegraded":"2024-09-15T23:58:12.150690+0000","last_fullsized":"2024-09-15T23:58:12.150690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:36.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.137347+0000","last_change":"2024-09-15T23:58:11.151745+0000","last_active":"2024-09-15T23:58:12.137347+0000","last_peered":"2024-09-15T23:58:12.137347+0000","last_clean":"2024-09-15T23:58:12.137347+0000","last_became_active":"2024-09-15T23:58:11.150864+0000","last_became_peered":"2024-09-15T23:58:11.150864+0000","last_unstale":"2024-09-15T23:58:12.137347+0000","last_undegraded":"2024-09-15T23:58:12.137347+0000","last_fullsized":"2024-09-15T23:58:12.137347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:34.578125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.300990+0000","last_change":"2024-09-15T23:58:09.121095+0000","last_active":"2024-09-15T23:58:23.300990+0000","last_peered":"2024-09-15T23:58:23.300990+0000","last_clean":"2024-09-15T23:58:23.300990+0000","last_became_active":"2024-09-15T23:58:09.120671+0000","last_became_peered":"2024-09-15T23:58:09.120671+0000","last_unstale":"2024-09-15T23:58:23.300990+0000","last_undegraded":"2024-09-15T23:58:23.300990+0000","last_fullsized":"2024-09-15T23:58:23.300990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:32.099810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.148041+0000","last_change":"2024-09-15T23:58:11.165256+0000","last_active":"2024-09-15T23:58:12.148041+0000","last_peered":"2024-09-15T23:58:12.148041+0000","last_clean":"2024-09-15T23:58:12.148041+0000","last_became_active":"2024-09-15T23:58:11.163663+0000","last_became_peered":"2024-09-15T23:58:11.163663+0000","last_unstale":"2024-09-15T23:58:12.148041+0000","last_undegraded":"2024-09-15T23:58:12.148041+0000","last_fullsized":"2024-09-15T23:58:12.148041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:02.112109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:23.934080+0000","last_change":"2024-09-15T23:58:09.125833+0000","last_active":"2024-09-15T23:58:23.934080+0000","last_peered":"2024-09-15T23:58:23.934080+0000","last_clean":"2024-09-15T23:58:23.934080+0000","last_became_active":"2024-09-15T23:58:09.125411+0000","last_became_peered":"2024-09-15T23:58:09.125411+0000","last_unstale":"2024-09-15T23:58:23.934080+0000","last_undegraded":"2024-09-15T23:58:23.934080+0000","last_fullsized":"2024-09-15T23:58:23.934080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:17.727261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.151115+0000","last_change":"2024-09-15T23:58:11.159923+0000","last_active":"2024-09-15T23:58:12.151115+0000","last_peered":"2024-09-15T23:58:12.151115+0000","last_clean":"2024-09-15T23:58:12.151115+0000","last_became_active":"2024-09-15T23:58:11.159489+0000","last_became_peered":"2024-09-15T23:58:11.159489+0000","last_unstale":"2024-09-15T23:58:12.151115+0000","last_undegraded":"2024-09-15T23:58:12.151115+0000","last_fullsized":"2024-09-15T23:58:12.151115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:30.864728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.153095+0000","last_change":"2024-09-15T23:58:09.322767+0000","last_active":"2024-09-15T23:58:12.153095+0000","last_peered":"2024-09-15T23:58:12.153095+0000","last_clean":"2024-09-15T23:58:12.153095+0000","last_became_active":"2024-09-15T23:58:09.322289+0000","last_became_peered":"2024-09-15T23:58:09.322289+0000","last_unstale":"2024-09-15T23:58:12.153095+0000","last_undegraded":"2024-09-15T23:58:12.153095+0000","last_fullsized":"2024-09-15T23:58:12.153095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:12.168772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:58:12.136810+0000","last_change":"2024-09-15T23:58:11.153181+0000","last_active":"2024-09-15T23:58:12.136810+0000","last_peered":"2024-09-15T23:58:12.136810+0000","last_clean":"2024-09-15T23:58:12.136810+0000","last_became_active":"2024-09-15T23:58:11.152920+0000","last_became_peered":"2024-09-15T23:58:11.152920+0000","last_unstale":"2024-09-15T23:58:12.136810+0000","last_undegraded":"2024-09-15T23:58:12.136810+0000","last_fullsized":"2024-09-15T23:58:12.136810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:10.116942+0000","last_clean_scrub_stamp":"2024-09-15T23:58:10.116942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:15.101157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":86,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-15T23:59:16.014915+0000","last_change":"2024-09-15T23:58:09.120470+0000","last_active":"2024-09-15T23:59:16.014915+0000","last_peered":"2024-09-15T23:59:16.014915+0000","last_clean":"2024-09-15T23:59:16.014915+0000","last_became_active":"2024-09-15T23:58:09.120143+0000","last_became_peered":"2024-09-15T23:58:09.120143+0000","last_unstale":"2024-09-15T23:59:16.014915+0000","last_undegraded":"2024-09-15T23:59:16.014915+0000","last_fullsized":"2024-09-15T23:59:16.014915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T23:58:08.103069+0000","last_clean_scrub_stamp":"2024-09-15T23:58:08.103069+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:41.285326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68567,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":572,"num_read_kb":364,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":205701,"data_compressed":5418,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244980,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1070174,"data_compressed":37506,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972390,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":417323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201323,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430255,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1010221,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593781,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1009506,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":416865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":479925,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":480098,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":418379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34060,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35700,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63897,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63724,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63488,"data_compressed":1806,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T23:59:20.877 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T23:59:20.877 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T23:59:20.877 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-15T23:59:20.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-15T23:59:21.140 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:21.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:21 smithi031 ceph-mon[31203]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:21 smithi102 ceph-mon[34458]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:21 smithi081 ceph-mon[35060]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.623+0000 7f19df2cd640 1 -- 172.21.15.31:0/3033175119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 msgr2=0x7f19d8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.623+0000 7f19df2cd640 1 --2- 172.21.15.31:0/3033175119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d8100cb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009900 tx=0x7f19d402a410 comp rx=0 tx=0).stop 2024-09-15T23:59:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.623+0000 7f19df2cd640 1 -- 172.21.15.31:0/3033175119 shutdown_connections 2024-09-15T23:59:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- 172.21.15.31:0/3033175119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d810e480 0x7f19d8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- 172.21.15.31:0/3033175119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d8101280 0x7f19d810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- 172.21.15.31:0/3033175119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d8100cb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- 172.21.15.31:0/3033175119 >> 172.21.15.31:0/3033175119 conn(0x7f19d80fc7a0 msgr2=0x7f19d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- 172.21.15.31:0/3033175119 shutdown_connections 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- 172.21.15.31:0/3033175119 wait complete. 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 Processor -- start 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- start start 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d8101280 0x7f19d819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d810e480 0x7f19d819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81140a0 con 0x7f19d81008d0 2024-09-15T23:59:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8113f20 con 0x7f19d810e480 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.624+0000 7f19df2cd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8114220 con 0x7f19d8101280 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58158/0 (socket says 172.21.15.31:58158) 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 -- 172.21.15.31:0/4244003419 learned_addr learned my addr 172.21.15.31:0/4244003419 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dc868640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d8101280 0x7f19d819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd86a640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d810e480 0x7f19d819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 -- 172.21.15.31:0/4244003419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d8101280 msgr2=0x7f19d819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d8101280 0x7f19d819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 -- 172.21.15.31:0/4244003419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d810e480 msgr2=0x7f19d819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d810e480 0x7f19d819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 -- 172.21.15.31:0/4244003419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d819f570 con 0x7f19d81008d0 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19dd069640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005940 tx=0x7f19d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:21.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d403f030 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d819f800 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d4003cb0 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4035070 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.625+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d81a7140 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.626+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d806b070 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.627+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f19d4042030 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.628+0000 7f19ce7fc640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 0x7f19c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.628+0000 7f19dc868640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 0x7f19c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.628+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f19d40f04c0 con 0x7f19d81008d0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.629+0000 7f19dc868640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 0x7f19c40a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19d810e8a0 tx=0x7f19c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:21.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.631+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19d40bd130 con 0x7f19d81008d0 2024-09-15T23:59:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.846+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f19d8100cb0 con 0x7f19d81008d0 2024-09-15T23:59:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.846+0000 7f19ce7fc640 1 -- 172.21.15.31:0/4244003419 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f19d8100cb0 con 0x7f19d81008d0 2024-09-15T23:59:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:21.847 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 msgr2=0x7f19c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 0x7f19c40a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19d810e8a0 tx=0x7f19c800b390 comp rx=0 tx=0).stop 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 msgr2=0x7f19d819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005940 tx=0x7f19d40042f0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 shutdown_connections 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f19c40a32d0 0x7f19c40a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19d810e480 0x7f19d819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19d8101280 0x7f19d819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 --2- 172.21.15.31:0/4244003419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d81008d0 0x7f19d819a5c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 >> 172.21.15.31:0/4244003419 conn(0x7f19d80fc7a0 msgr2=0x7f19d80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 shutdown_connections 2024-09-15T23:59:21.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:21.849+0000 7f19df2cd640 1 -- 172.21.15.31:0/4244003419 wait complete. 2024-09-15T23:59:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:22 smithi031 ceph-mon[31203]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:22.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:22 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/4244003419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T23:59:22.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:22 smithi102 ceph-mon[34458]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:22.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:22 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/4244003419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T23:59:22.540 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-15T23:59:22.540 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:59:22.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:22 smithi081 ceph-mon[35060]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T23:59:22.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:22 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/4244003419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T23:59:22.800 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:23.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:23 smithi031 ceph-mon[31203]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 -- 172.21.15.31:0/3862589857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc106f10 msgr2=0x7f9bbc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/3862589857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc106f10 0x7f9bbc10ea90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009ca0 tx=0x7f9bb402a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 -- 172.21.15.31:0/3862589857 shutdown_connections 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/3862589857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc106f10 0x7f9bbc10ea90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/3862589857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc102790 0x7f9bbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/3862589857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc101de0 0x7f9bbc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 -- 172.21.15.31:0/3862589857 >> 172.21.15.31:0/3862589857 conn(0x7f9bbc0fd8c0 msgr2=0x7f9bbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 -- 172.21.15.31:0/3862589857 shutdown_connections 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 -- 172.21.15.31:0/3862589857 wait complete. 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.317+0000 7f9bc14b4640 1 Processor -- start 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 -- start start 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc102790 0x7f9bbc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 0x7f9bbc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc111e40 con 0x7f9bbc101de0 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc111cc0 con 0x7f9bbc106f10 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bc14b4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc111fc0 con 0x7f9bbc102790 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58174/0 (socket says 172.21.15.31:58174) 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 -- 172.21.15.31:0/2826232723 learned_addr learned my addr 172.21.15.31:0/2826232723 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbb7fe640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 0x7f9bbc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bba7fc640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc102790 0x7f9bbc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 -- 172.21.15.31:0/2826232723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc102790 msgr2=0x7f9bbc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc102790 0x7f9bbc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 -- 172.21.15.31:0/2826232723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 msgr2=0x7f9bbc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 0x7f9bbc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 -- 172.21.15.31:0/2826232723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc1a67e0 con 0x7f9bbc101de0 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbb7fe640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 0x7f9bbc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.318+0000 7f9bbaffd640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00b4a0 tx=0x7f9bac00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.319+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac015020 con 0x7f9bbc101de0 2024-09-15T23:59:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.319+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc1a6ad0 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.319+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bac011ce0 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.319+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac010dc0 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.319+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc1a7010 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.320+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bbc1021c0 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.321+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9bac018020 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.322+0000 7f9ba3fff640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 0x7f9ba40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.322+0000 7f9bba7fc640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 0x7f9ba40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.322+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9bac0cf710 con 0x7f9bbc101de0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.322+0000 7f9bba7fc640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 0x7f9ba40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc075f80 tx=0x7f9bb000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.325+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bac01f020 con 0x7f9bbc101de0 2024-09-15T23:59:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:23 smithi102 ceph-mon[34458]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:23.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:23 smithi081 ceph-mon[35060]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:23.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.536+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bbc075a20 con 0x7f9bbc101de0 2024-09-15T23:59:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.536+0000 7f9ba3fff640 1 -- 172.21.15.31:0/2826232723 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7f9bbc075a20 con 0x7f9bbc101de0 2024-09-15T23:59:23.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:23.538 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:59:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 msgr2=0x7f9ba40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 0x7f9ba40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc075f80 tx=0x7f9bb000a2d0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 msgr2=0x7f9bbc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00b4a0 tx=0x7f9bac00b8a0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 shutdown_connections 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9ba40a32d0 0x7f9ba40a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bbc106f10 0x7f9bbc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bbc102790 0x7f9bbc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 --2- 172.21.15.31:0/2826232723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bbc101de0 0x7f9bbc0744f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 >> 172.21.15.31:0/2826232723 conn(0x7f9bbc0fd8c0 msgr2=0x7f9bbc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 shutdown_connections 2024-09-15T23:59:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:23.539+0000 7f9bc14b4640 1 -- 172.21.15.31:0/2826232723 wait complete. 2024-09-15T23:59:24.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:24 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2826232723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:24.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:59:24.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:24 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2826232723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:24.537 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:24.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:24 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2826232723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 -- 172.21.15.31:0/1255013728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec101de0 msgr2=0x7f40ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 --2- 172.21.15.31:0/1255013728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec101de0 0x7f40ec1021c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009900 tx=0x7f40dc02a410 comp rx=0 tx=0).stop 2024-09-15T23:59:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 -- 172.21.15.31:0/1255013728 shutdown_connections 2024-09-15T23:59:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 --2- 172.21.15.31:0/1255013728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec106f10 0x7f40ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 --2- 172.21.15.31:0/1255013728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec102790 0x7f40ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 --2- 172.21.15.31:0/1255013728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec101de0 0x7f40ec1021c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 -- 172.21.15.31:0/1255013728 >> 172.21.15.31:0/1255013728 conn(0x7f40ec0fd8c0 msgr2=0x7f40ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 -- 172.21.15.31:0/1255013728 shutdown_connections 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.006+0000 7f40f29c1640 1 -- 172.21.15.31:0/1255013728 wait complete. 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 Processor -- start 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 -- start start 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 0x7f40ec198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec106f10 0x7f40ec19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec111ec0 con 0x7f40ec102790 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec111d40 con 0x7f40ec106f10 2024-09-15T23:59:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40f29c1640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec112040 con 0x7f40ec101de0 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.007+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58188/0 (socket says 172.21.15.31:58188) 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 -- 172.21.15.31:0/81204580 learned_addr learned my addr 172.21.15.31:0/81204580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40f0f5e640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec106f10 0x7f40ec19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40ebfff640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 0x7f40ec198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 -- 172.21.15.31:0/81204580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 msgr2=0x7f40ec198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 0x7f40ec198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 -- 172.21.15.31:0/81204580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec106f10 msgr2=0x7f40ec19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec106f10 0x7f40ec19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 -- 172.21.15.31:0/81204580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec19d4d0 con 0x7f40ec102790 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40ebfff640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 0x7f40ec198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40eb7fe640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b4a0 tx=0x7f40e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0014070 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec19d7c0 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e0010ac0 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0017c30 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.008+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a50a0 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.009+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec102240 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.010+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f40e0017070 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.011+0000 7f40e97fa640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 0x7f40d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.011+0000 7f40ebfff640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 0x7f40d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.011+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f40e00ceec0 con 0x7f40ec102790 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.012+0000 7f40ebfff640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 0x7f40d40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40dc031440 tx=0x7f40dc0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.015+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40e009bd40 con 0x7f40ec102790 2024-09-15T23:59:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.227+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40ec109b60 con 0x7f40ec102790 2024-09-15T23:59:25.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:25 smithi031 ceph-mon[31203]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.227+0000 7f40e97fa640 1 -- 172.21.15.31:0/81204580 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7f40e0097070 con 0x7f40ec102790 2024-09-15T23:59:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:25.229 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:59:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 msgr2=0x7f40d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 0x7f40d40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40dc031440 tx=0x7f40dc0095f0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 msgr2=0x7f40ec198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b4a0 tx=0x7f40e000b960 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 shutdown_connections 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f40d40a32d0 0x7f40d40a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec106f10 0x7f40ec19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ec102790 0x7f40ec198a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 --2- 172.21.15.31:0/81204580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ec101de0 0x7f40ec198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 >> 172.21.15.31:0/81204580 conn(0x7f40ec0fd8c0 msgr2=0x7f40ec0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 shutdown_connections 2024-09-15T23:59:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:25.230+0000 7f40f29c1640 1 -- 172.21.15.31:0/81204580 wait complete. 2024-09-15T23:59:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:25 smithi102 ceph-mon[34458]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:25.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:25 smithi081 ceph-mon[35060]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:25.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:59:26.217 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:26.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:26 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/81204580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:26.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:26 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/81204580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:26.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:26 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/81204580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 -- 172.21.15.31:0/951047858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 msgr2=0x7fe48810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 --2- 172.21.15.31:0/951047858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48810ec70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe480009cd0 tx=0x7fe48002a8d0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 -- 172.21.15.31:0/951047858 shutdown_connections 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 --2- 172.21.15.31:0/951047858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48810ec70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 --2- 172.21.15.31:0/951047858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 0x7fe488107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 --2- 172.21.15.31:0/951047858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 0x7fe4881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 -- 172.21.15.31:0/951047858 >> 172.21.15.31:0/951047858 conn(0x7fe4880fc7d0 msgr2=0x7fe4880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 -- 172.21.15.31:0/951047858 shutdown_connections 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.731+0000 7fe48eac0640 1 -- 172.21.15.31:0/951047858 wait complete. 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 Processor -- start 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 -- start start 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 0x7fe4881984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 0x7fe4881989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111f80 con 0x7fe488108300 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111e00 con 0x7fe488103780 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48eac0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112100 con 0x7fe488102dd0 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58200/0 (socket says 172.21.15.31:58200) 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 -- 172.21.15.31:0/1408267311 learned_addr learned my addr 172.21.15.31:0/1408267311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe487fff640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 0x7fe4881989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48c85c640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 0x7fe4881984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 -- 172.21.15.31:0/1408267311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 msgr2=0x7fe4881984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 0x7fe4881984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 -- 172.21.15.31:0/1408267311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 msgr2=0x7fe4881989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 0x7fe4881989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 -- 172.21.15.31:0/1408267311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48819d450 con 0x7fe488108300 2024-09-15T23:59:26.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.732+0000 7fe48d05d640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe480005cc0 tx=0x7fe480032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.733+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480003b70 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.733+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe480003d10 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.733+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800307d0 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.733+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48819d6e0 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.733+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a5020 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.734+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48806a7b0 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.735+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe4800309f0 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.736+0000 7fe485ffb640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 0x7fe4740a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.736+0000 7fe48c85c640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 0x7fe4740a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.736+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe4800f08e0 con 0x7fe488108300 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.736+0000 7fe48c85c640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 0x7fe4740a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe478005fd0 tx=0x7fe478005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:26.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.740+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe480030ce0 con 0x7fe488108300 2024-09-15T23:59:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.916+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe48810a290 con 0x7fe488108300 2024-09-15T23:59:26.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.917+0000 7fe485ffb640 1 -- 172.21.15.31:0/1408267311 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7fe4800bd660 con 0x7fe488108300 2024-09-15T23:59:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:26.919 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 msgr2=0x7fe4740a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 0x7fe4740a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe478005fd0 tx=0x7fe478005850 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 msgr2=0x7fe48819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe480005cc0 tx=0x7fe480032cf0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 shutdown_connections 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fe4740a3350 0x7fe4740a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488108300 0x7fe48819cd70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488103780 0x7fe4881989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 --2- 172.21.15.31:0/1408267311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe488102dd0 0x7fe4881984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 >> 172.21.15.31:0/1408267311 conn(0x7fe4880fc7d0 msgr2=0x7fe488105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 shutdown_connections 2024-09-15T23:59:26.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:26.920+0000 7fe48eac0640 1 -- 172.21.15.31:0/1408267311 wait complete. 2024-09-15T23:59:27.231 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:27 smithi031 ceph-mon[31203]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:27.231 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:27 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1408267311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:27 smithi102 ceph-mon[34458]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:27.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:27 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1408267311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:27.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:27 smithi081 ceph-mon[35060]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:27.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:27 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1408267311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:27.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:59:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- 172.21.15.31:0/869864322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 msgr2=0x7f7920103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 --2- 172.21.15.31:0/869864322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f7920103170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7910009930 tx=0x7f791002a420 comp rx=0 tx=0).stop 2024-09-15T23:59:28.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- 172.21.15.31:0/869864322 shutdown_connections 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 --2- 172.21.15.31:0/869864322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920108290 0x7f792010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 --2- 172.21.15.31:0/869864322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920103740 0x7f7920107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 --2- 172.21.15.31:0/869864322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f7920103170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- 172.21.15.31:0/869864322 >> 172.21.15.31:0/869864322 conn(0x7f79200fc7d0 msgr2=0x7f79200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- 172.21.15.31:0/869864322 shutdown_connections 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- 172.21.15.31:0/869864322 wait complete. 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 Processor -- start 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.376+0000 7f79266be640 1 -- start start 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920103740 0x7f7920100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 0x7f7920101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7920111f40 con 0x7f7920102d90 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7920111dc0 con 0x7f7920103740 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f79266be640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f79201120c0 con 0x7f7920108290 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58214/0 (socket says 172.21.15.31:58214) 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 -- 172.21.15.31:0/3677135438 learned_addr learned my addr 172.21.15.31:0/3677135438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f7924c5b640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 0x7f7920101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791f7fe640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920103740 0x7f7920100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 -- 172.21.15.31:0/3677135438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 msgr2=0x7f7920101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 0x7f7920101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 -- 172.21.15.31:0/3677135438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920103740 msgr2=0x7f7920100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920103740 0x7f7920100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f791ffff640 1 -- 172.21.15.31:0/3677135438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79201a69c0 con 0x7f7920102d90 2024-09-15T23:59:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.377+0000 7f7924c5b640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 0x7f7920101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f791ffff640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7910004770 tx=0x7f79100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791003f070 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f791003b460 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79201a6c50 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910035070 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.378+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79201a7130 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.379+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792006a7f0 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.380+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f791003b610 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.381+0000 7f791d7fa640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 0x7f79080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.381+0000 7f791f7fe640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 0x7f79080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.381+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f79100efe30 con 0x7f7920102d90 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.382+0000 7f791f7fe640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 0x7f79080a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7920102480 tx=0x7f791400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.385+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79100bcaa0 con 0x7f7920102d90 2024-09-15T23:59:28.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.594+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792010a2a0 con 0x7f7920102d90 2024-09-15T23:59:28.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.595+0000 7f791d7fa640 1 -- 172.21.15.31:0/3677135438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7f79100bc3a0 con 0x7f7920102d90 2024-09-15T23:59:28.596 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:28.596 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:59:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 msgr2=0x7f79080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 0x7f79080a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7920102480 tx=0x7f791400b590 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 msgr2=0x7f79201009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7910004770 tx=0x7f79100042f0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 shutdown_connections 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f79080a32d0 0x7f79080a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7920108290 0x7f7920101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7920103740 0x7f7920100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 --2- 172.21.15.31:0/3677135438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920102d90 0x7f79201009f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 >> 172.21.15.31:0/3677135438 conn(0x7f79200fc7d0 msgr2=0x7f7920105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 shutdown_connections 2024-09-15T23:59:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:28.598+0000 7f79266be640 1 -- 172.21.15.31:0/3677135438 wait complete. 2024-09-15T23:59:29.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T23:58:11.769531+0000', 'modified': '2024-09-15T23:58:23.979593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24491, 'mds_1': 24478, 'mds_2': 24461, 'mds_3': 24457, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/193010383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 193010383}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 193010383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14607': {'gid': 14607, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/1341414618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 1341414618}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 1341414618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/958737504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 958737504}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 958737504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6833/4135774449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 4135774449}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 4135774449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6835/799148008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 799148008}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 799148008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.102:6835/2759418980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2759418980}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2759418980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.102:6837/4082772592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 4082772592}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 4082772592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.81:6837/3184710981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3184710981}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3184710981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.102:6839/3686122257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3686122257}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3686122257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/471643993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 471643993}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 471643993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24491, 24478, 24461, 24457, 24451]} 2024-09-15T23:59:29.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T23:59:29.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-15T23:59:29.287 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:29 smithi031 ceph-mon[31203]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:29.287 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:29 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3677135438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:29 smithi102 ceph-mon[34458]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:29 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3677135438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:29.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:29 smithi081 ceph-mon[35060]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:29.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:29 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3677135438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 -- 172.21.15.31:0/2997202989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468102570 msgr2=0x7f74681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 --2- 172.21.15.31:0/2997202989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468102570 0x7f74681029b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ca0 tx=0x7f745c02a950 comp rx=0 tx=0).stop 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 -- 172.21.15.31:0/2997202989 shutdown_connections 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 --2- 172.21.15.31:0/2997202989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 0x7f746810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 --2- 172.21.15.31:0/2997202989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468102570 0x7f74681029b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.977+0000 7f746d44b640 1 --2- 172.21.15.31:0/2997202989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468100e20 0x7f7468101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 -- 172.21.15.31:0/2997202989 >> 172.21.15.31:0/2997202989 conn(0x7f74680fc7d0 msgr2=0x7f74680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 -- 172.21.15.31:0/2997202989 shutdown_connections 2024-09-15T23:59:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 -- 172.21.15.31:0/2997202989 wait complete. 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 Processor -- start 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 -- start start 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.978+0000 7f746d44b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468102570 0x7f746810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 0x7f746810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7468111f50 con 0x7f7468100e20 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7468111dd0 con 0x7f7468102570 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f74681120d0 con 0x7f7468102ef0 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58234/0 (socket says 172.21.15.31:58234) 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 -- 172.21.15.31:0/1832193958 learned_addr learned my addr 172.21.15.31:0/1832193958 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f74667fc640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468102570 0x7f746810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f74677fe640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 0x7f746810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 -- 172.21.15.31:0/1832193958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 msgr2=0x7f746810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 0x7f746810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 -- 172.21.15.31:0/1832193958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468102570 msgr2=0x7f746810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468102570 0x7f746810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 -- 172.21.15.31:0/1832193958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74681a6900 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f745800c920 tx=0x7f745800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f744ffff640 1 -- 172.21.15.31:0/1832193958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458007c70 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f744ffff640 1 -- 172.21.15.31:0/1832193958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7458007e10 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681a6bf0 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f744ffff640 1 -- 172.21.15.31:0/1832193958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458018820 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.979+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a7180 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f744ffff640 1 -- 172.21.15.31:0/1832193958 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f74580189c0 con 0x7f7468100e20 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 Processor -- start 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 -- start start 2024-09-15T23:59:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652d910a090 con 0x5652d91aebb0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652d90b1a70 con 0x5652d91a42f0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f746f824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5652d9106050 con 0x5652d91a39d0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f74667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f74667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58248/0 (socket says 172.21.15.31:58248) 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f74667fc640 1 -- 172.21.15.31:0/2277242099 learned_addr learned my addr 172.21.15.31:0/2277242099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.982+0000 7f74677fe640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f7466ffd640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f744ffff640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 0x7f74500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 -- 172.21.15.31:0/2277242099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 msgr2=0x5652d91a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 -- 172.21.15.31:0/2277242099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 msgr2=0x5652d91ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 -- 172.21.15.31:0/2277242099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d90be130 con 0x5652d91aebb0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74677fe640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 0x7f74500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f7466ffd640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f744ffff640 1 -- 172.21.15.31:0/1832193958 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f74580cfb70 con 0x7f7468100e20 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74667fc640 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f745c035000 tx=0x7f745c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc0a709cf2d264f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f74677fe640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 0x7f74500a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7460005d80 tx=0x7f74600046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f744d7fa640 1 -- 172.21.15.31:0/2277242099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c034040 con 0x5652d91aebb0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f744d7fa640 1 -- 172.21.15.31:0/2277242099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745c003970 con 0x5652d91aebb0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.983+0000 7f744d7fa640 1 -- 172.21.15.31:0/2277242099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c0045e0 con 0x5652d91aebb0 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- 172.21.15.31:0/2277242099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 msgr2=0x5652d91b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f745c035000 tx=0x7f745c009a20 comp rx=0 tx=0).stop 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- 172.21.15.31:0/2277242099 shutdown_connections 2024-09-15T23:59:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91aebb0 0x5652d91b0fa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 --2- 172.21.15.31:0/2277242099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- 172.21.15.31:0/2277242099 >> 172.21.15.31:0/2277242099 conn(0x5652d919ff60 msgr2=0x5652d91a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- 172.21.15.31:0/2277242099 shutdown_connections 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- 172.21.15.31:0/2277242099 wait complete. 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.984+0000 7f746f824740 1 -- start start 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 Processor -- start 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652d91b1780 con 0x5652d91a39d0 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652d90c4f20 con 0x5652d91a42f0 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5652d91b1900 con 0x5652d91aebb0 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f746f824740 10 client.0 _ll_register_callbacks cb 0x5652d908e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58262/0 (socket says 172.21.15.31:58262) 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 -- 172.21.15.31:0/3477485154 learned_addr learned my addr 172.21.15.31:0/3477485154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f74677fe640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f74667fc640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 -- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 msgr2=0x5652d91abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 -- 172.21.15.31:0/3477485154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 msgr2=0x5652d91ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 -- 172.21.15.31:0/3477485154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d8e7b2b0 con 0x5652d91a39d0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f74667fc640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f7466ffd640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f745809bde0 tx=0x7f745809bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7d7d25935214de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.985+0000 7f744aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800de20 con 0x5652d91a39d0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f74580a0120 con 0x5652d91a39d0 2024-09-15T23:59:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c003970 con 0x5652d91a39d0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f745809c7e0 con 0x5652d91a39d0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 10 client.14964 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652d91bd060 con 0x5652d91a39d0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745c003970 con 0x5652d91a39d0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f74580a48c0 con 0x5652d91a39d0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 10 client.14964 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 10 client.14964 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 20 client.14964 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d91c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 4 client.14964 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.986+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 --> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652d9117600 con 0x5652d91bf640 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.987+0000 7f74667fc640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d91c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.987+0000 7f74667fc640 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d91c0ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66d6d436a8b2ecfc server_cookie=1ec8a97214b6f3d4 in_seq=0 out_seq=0 2024-09-15T23:59:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.987+0000 7f744aff5640 10 client.14964 ms_handle_connect on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f744aff5640 1 -- 172.21.15.31:0/3477485154 <== mds.4 v2:172.21.15.102:6834/2759418980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652d9117600 con 0x5652d91bf640 2024-09-15T23:59:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f744aff5640 10 client.14964 handle_command_reply: tid=0 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 client.14964 shutdown 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 msgr2=0x5652d91ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f745809bde0 tx=0x7f745809bb00 comp rx=0 tx=0).stop 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 shutdown_connections 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d91c0ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f744aff5640 0 client.14964 ms_handle_reset on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 --2- 172.21.15.31:0/3477485154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91ae390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 >> 172.21.15.31:0/3477485154 conn(0x5652d919ff60 msgr2=0x5652d91a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 shutdown_connections 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 1 -- 172.21.15.31:0/3477485154 wait complete. 2024-09-15T23:59:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:29.988+0000 7f746f824740 20 client.14964 trim_cache size 0 max 16384 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 Processor -- start 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 -- start start 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91aebb0 0x5652d91abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652d91bedf0 con 0x5652d91a42f0 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652d91bf4c0 con 0x5652d91aebb0 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f746f824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5652d91bfb90 con 0x5652d91a39d0 2024-09-15T23:59:30.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.006+0000 7f74667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58264/0 (socket says 172.21.15.31:58264) 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 -- 172.21.15.31:0/1418545060 learned_addr learned my addr 172.21.15.31:0/1418545060 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 -- 172.21.15.31:0/1418545060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 msgr2=0x5652d91bef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74677fe640 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91aebb0 0x5652d91abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91bef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 -- 172.21.15.31:0/1418545060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91aebb0 msgr2=0x5652d91abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91aebb0 0x5652d91abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 -- 172.21.15.31:0/1418545060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d91a4120 con 0x5652d91a42f0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f74667fc640 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f745c004190 tx=0x7f745c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1468d6cc8c391333 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f744cff9640 1 -- 172.21.15.31:0/1418545060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c034040 con 0x5652d91a42f0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f744cff9640 1 -- 172.21.15.31:0/1418545060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745c052070 con 0x5652d91a42f0 2024-09-15T23:59:30.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.007+0000 7f744cff9640 1 -- 172.21.15.31:0/1418545060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c004230 con 0x5652d91a42f0 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- 172.21.15.31:0/1418545060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 msgr2=0x5652d91ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f745c004190 tx=0x7f745c005fd0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- 172.21.15.31:0/1418545060 shutdown_connections 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91aebb0 0x5652d91abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a42f0 0x5652d91ab6a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- 172.21.15.31:0/1418545060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91a39d0 0x5652d91bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- 172.21.15.31:0/1418545060 >> 172.21.15.31:0/1418545060 conn(0x5652d919ff60 msgr2=0x5652d919ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- 172.21.15.31:0/1418545060 shutdown_connections 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- 172.21.15.31:0/1418545060 wait complete. 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 -- start start 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 Processor -- start 2024-09-15T23:59:30.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.008+0000 7f746f824740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91a8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652d91c0930 con 0x5652d91a39d0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652d91c0260 con 0x5652d91a42f0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5652d91c1000 con 0x5652d91aebb0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 10 client.0 _ll_register_callbacks cb 0x7f745800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58266/0 (socket says 172.21.15.31:58266) 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 -- 172.21.15.31:0/1284758076 learned_addr learned my addr 172.21.15.31:0/1284758076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f74667fc640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f74677fe640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 -- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 msgr2=0x5652d91a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 -- 172.21.15.31:0/1284758076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 msgr2=0x5652d91a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91a8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 -- 172.21.15.31:0/1284758076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d91c7e80 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f74667fc640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f7466ffd640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7458007590 tx=0x7f74580bc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5706bff99cf4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f744aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458098380 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f74580a48c0 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c052070 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.009+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f745800de20 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 10 client.14970 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652d91c9120 con 0x5652d91a39d0 2024-09-15T23:59:30.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745c052070 con 0x5652d91a39d0 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f745809afa0 con 0x5652d91a39d0 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 10 client.14970 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 10 client.14970 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 20 client.14970 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d922c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 4 client.14970 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 --> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] -- command(tid 0: {"prefix": "status"}) -- 0x5652d9111b10 con 0x5652d91bf640 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.010+0000 7f74667fc640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d922c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f74667fc640 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d922c940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7926ba4dd6898910 server_cookie=268920534f30891 in_seq=0 out_seq=0 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f744aff5640 10 client.14970 ms_handle_connect on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f744aff5640 1 -- 172.21.15.31:0/1284758076 <== mds.4 v2:172.21.15.102:6834/2759418980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5652d9111b10 con 0x5652d91bf640 2024-09-15T23:59:30.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f744aff5640 10 client.14970 handle_command_reply: tid=0 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 client.14970 shutdown 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 msgr2=0x5652d91a7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7458007590 tx=0x7f74580bc2e0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 shutdown_connections 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5652d91bf640 0x5652d922c940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652d91aebb0 0x5652d91a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652d91a42f0 0x5652d91a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f744aff5640 0 client.14970 ms_handle_reset on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 --2- 172.21.15.31:0/1284758076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652d91a39d0 0x5652d91a7af0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 >> 172.21.15.31:0/1284758076 conn(0x5652d919ff60 msgr2=0x5652d91ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:30.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 shutdown_connections 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 1 -- 172.21.15.31:0/1284758076 wait complete. 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.011+0000 7f746f824740 20 client.14970 trim_cache size 0 max 16384 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24451, 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 66.721660713000006, 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 105.756179725 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 msgr2=0x7f74500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 0x7f74500a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7460005d80 tx=0x7f74600046c0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 msgr2=0x7f746810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f745800c920 tx=0x7f745800cde0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 shutdown_connections 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f74500a3350 0x7f74500a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7468102ef0 0x7f746810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7468102570 0x7f746810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 --2- 172.21.15.31:0/1832193958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7468100e20 0x7f746810e4d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 >> 172.21.15.31:0/1832193958 conn(0x7f74680fc7d0 msgr2=0x7f7468106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 shutdown_connections 2024-09-15T23:59:30.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:30.012+0000 7f746d44b640 1 -- 172.21.15.31:0/1832193958 wait complete. 2024-09-15T23:59:30.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-15T23:59:31.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:31 smithi102 ceph-mon[34458]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:31.421 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:31 smithi031 ceph-mon[31203]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 -- 172.21.15.31:0/1680646398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c10e4f0 msgr2=0x7f566c1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 --2- 172.21.15.31:0/1680646398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c10e4f0 0x7f566c1108e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 -- 172.21.15.31:0/1680646398 shutdown_connections 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 --2- 172.21.15.31:0/1680646398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c10e4f0 0x7f566c1108e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 --2- 172.21.15.31:0/1680646398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c1012f0 0x7f566c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 --2- 172.21.15.31:0/1680646398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 0x7f566c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.503+0000 7f56719a7640 1 -- 172.21.15.31:0/1680646398 >> 172.21.15.31:0/1680646398 conn(0x7f566c0fc790 msgr2=0x7f566c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 -- 172.21.15.31:0/1680646398 shutdown_connections 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 -- 172.21.15.31:0/1680646398 wait complete. 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 Processor -- start 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 -- start start 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 0x7f566c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.504+0000 7f56719a7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c10e4f0 0x7f566c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56719a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f566c114070 con 0x7f566c1012f0 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56719a7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f566c113ef0 con 0x7f566c10e4f0 2024-09-15T23:59:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56719a7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1141f0 con 0x7f566c100940 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58278/0 (socket says 172.21.15.31:58278) 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 -- 172.21.15.31:0/1027007297 learned_addr learned my addr 172.21.15.31:0/1027007297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566affd640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 0x7f566c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566b7fe640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c10e4f0 0x7f566c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 -- 172.21.15.31:0/1027007297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 msgr2=0x7f566c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 0x7f566c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 -- 172.21.15.31:0/1027007297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c10e4f0 msgr2=0x7f566c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c10e4f0 0x7f566c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 -- 172.21.15.31:0/1027007297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c19f5a0 con 0x7f566c1012f0 2024-09-15T23:59:31.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f566000d950 tx=0x7f566000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56709a5640 1 -- 172.21.15.31:0/1027007297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660017070 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56709a5640 1 -- 172.21.15.31:0/1027007297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5660009c30 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c19f890 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56709a5640 1 -- 172.21.15.31:0/1027007297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566000f620 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.505+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a7170 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f56709a5640 1 -- 172.21.15.31:0/1027007297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f56600109a0 con 0x7f566c1012f0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 Processor -- start 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 -- start start 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3ea9090 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3e50a70 con 0x55a7f3f429d0 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f5673d80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3ea5050 con 0x55a7f3f432f0 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58280/0 (socket says 172.21.15.31:58280) 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f566a7fc640 1 -- 172.21.15.31:0/613677172 learned_addr learned my addr 172.21.15.31:0/613677172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.508+0000 7f566b7fe640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566affd640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f56709a5640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 0x7f56540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 -- 172.21.15.31:0/613677172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 msgr2=0x55a7f3f4d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 -- 172.21.15.31:0/613677172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 msgr2=0x55a7f3f42db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f42db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 -- 172.21.15.31:0/613677172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f3e5d130 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566affd640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566b7fe640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 0x7f56540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f56709a5640 1 -- 172.21.15.31:0/1027007297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f56600cf670 con 0x7f566c1012f0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566a7fc640 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5660069bb0 tx=0x7f566006a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c9d0aa741f4ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f5651ffb640 1 -- 172.21.15.31:0/613677172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566009b720 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f566b7fe640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 0x7f56540a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f5651ffb640 1 -- 172.21.15.31:0/613677172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5660069c10 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.509+0000 7f5651ffb640 1 -- 172.21.15.31:0/613677172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660069e70 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- 172.21.15.31:0/613677172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 msgr2=0x55a7f3f4ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5660069bb0 tx=0x7f566006a920 comp rx=0 tx=0).stop 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- 172.21.15.31:0/613677172 shutdown_connections 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ffa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 --2- 172.21.15.31:0/613677172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f42db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- 172.21.15.31:0/613677172 >> 172.21.15.31:0/613677172 conn(0x55a7f3f3ef60 msgr2=0x55a7f3f3f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- 172.21.15.31:0/613677172 shutdown_connections 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- 172.21.15.31:0/613677172 wait complete. 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.510+0000 7f5673d80740 1 -- start start 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 Processor -- start 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f50780 con 0x55a7f3f429d0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3e63f20 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f50900 con 0x55a7f3f432f0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f5673d80740 10 client.0 _ll_register_callbacks cb 0x55a7f3e2d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58294/0 (socket says 172.21.15.31:58294) 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 -- 172.21.15.31:0/1014361602 learned_addr learned my addr 172.21.15.31:0/1014361602 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566b7fe640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 -- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 msgr2=0x55a7f3f4a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 -- 172.21.15.31:0/1014361602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f4dbb0 msgr2=0x55a7f3f4ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.511+0000 7f566affd640 1 -- 172.21.15.31:0/1014361602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f3c1a2b0 con 0x55a7f3f429d0 2024-09-15T23:59:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f566affd640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f565c00af30 tx=0x7f565c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fa3685e63b5e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f564affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01b060 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7f3eb6600 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f565c016640 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f565c016840 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f5673d80740 10 client.14982 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7f3f5c0f0 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.512+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a7f3eb6600 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f565c0153e0 con 0x55a7f3f429d0 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 10 client.14982 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 20 client.14982 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3f5e6c0 0x55a7f3f5fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 4 client.14982 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 --> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f3eb0b10 con 0x55a7f3f5e6c0 2024-09-15T23:59:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3f5e6c0 0x55a7f3f5fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f566a7fc640 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3f5e6c0 0x55a7f3f5fb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47ad32af590183b4 server_cookie=7c4f54dc075ce359 in_seq=0 out_seq=0 2024-09-15T23:59:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.513+0000 7f564affd640 10 client.14982 ms_handle_connect on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f564affd640 1 -- 172.21.15.31:0/1014361602 <== mds.3 v2:172.21.15.102:6836/4082772592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7f3eb0b10 con 0x55a7f3f5e6c0 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f564affd640 10 client.14982 handle_command_reply: tid=0 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 client.14982 shutdown 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 msgr2=0x55a7f3f4d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f565c00af30 tx=0x7f565c007680 comp rx=0 tx=0).stop 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 shutdown_connections 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f4ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3f5e6c0 0x55a7f3f5fb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f432f0 0x55a7f3f4a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 --2- 172.21.15.31:0/1014361602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f429d0 0x55a7f3f4d3f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 >> 172.21.15.31:0/1014361602 conn(0x55a7f3f3ef60 msgr2=0x55a7f3f44c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f564affd640 0 client.14982 ms_handle_reset on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.514+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 shutdown_connections 2024-09-15T23:59:31.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.515+0000 7f5673d80740 1 -- 172.21.15.31:0/1014361602 wait complete. 2024-09-15T23:59:31.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.515+0000 7f5673d80740 20 client.14982 trim_cache size 0 max 16384 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.532+0000 7f5673d80740 1 Processor -- start 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.532+0000 7f5673d80740 1 -- start start 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f5fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f3def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f5de70 con 0x55a7f3f432f0 2024-09-15T23:59:31.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f5e540 con 0x55a7f3f429d0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f5673d80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f5ec10 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58304/0 (socket says 172.21.15.31:58304) 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 -- 172.21.15.31:0/2745795950 learned_addr learned my addr 172.21.15.31:0/2745795950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566affd640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f5fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566b7fe640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f3def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 -- 172.21.15.31:0/2745795950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 msgr2=0x55a7f3f3def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f3def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 -- 172.21.15.31:0/2745795950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 msgr2=0x55a7f3f5fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f5fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 -- 172.21.15.31:0/2745795950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f3f4d760 con 0x55a7f3f432f0 2024-09-15T23:59:31.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566b7fe640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f3def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566affd640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f5fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f566a7fc640 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f56600a4320 tx=0x7f56600cf8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58520dfd84e224f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f56517fa640 1 -- 172.21.15.31:0/2745795950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660020040 con 0x55a7f3f432f0 2024-09-15T23:59:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f56517fa640 1 -- 172.21.15.31:0/2745795950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f566009bbe0 con 0x55a7f3f432f0 2024-09-15T23:59:31.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.533+0000 7f56517fa640 1 -- 172.21.15.31:0/2745795950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566009b540 con 0x55a7f3f432f0 2024-09-15T23:59:31.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- 172.21.15.31:0/2745795950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 msgr2=0x55a7f3f3f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f56600a4320 tx=0x7f56600cf8f0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- 172.21.15.31:0/2745795950 shutdown_connections 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3f3def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f3f340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 --2- 172.21.15.31:0/2745795950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f5fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- 172.21.15.31:0/2745795950 >> 172.21.15.31:0/2745795950 conn(0x55a7f3f3ef60 msgr2=0x55a7f3f3d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- 172.21.15.31:0/2745795950 shutdown_connections 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- 172.21.15.31:0/2745795950 wait complete. 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.534+0000 7f5673d80740 1 -- start start 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 Processor -- start 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3fc6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f5f9b0 con 0x55a7f3f432f0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f5f2e0 con 0x55a7f3f429d0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3f60080 con 0x55a7f3f4dbb0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f5673d80740 10 client.0 _ll_register_callbacks cb 0x7f565c0098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58320/0 (socket says 172.21.15.31:58320) 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 -- 172.21.15.31:0/364876382 learned_addr learned my addr 172.21.15.31:0/364876382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566affd640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566b7fe640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3fc6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 -- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 msgr2=0x55a7f3fc6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3fc6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 -- 172.21.15.31:0/364876382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 msgr2=0x55a7f3f632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.535+0000 7f566a7fc640 1 -- 172.21.15.31:0/364876382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f3fc7400 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f566a7fc640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f566009bb70 tx=0x7f56600cd090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6a8a182777d7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660020070 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5660098250 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566009bbe0 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f566009ff90 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 10 client.14988 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7f3fc8650 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56600b2080 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f566009be40 con 0x55a7f3f432f0 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 10 client.14988 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 10 client.14988 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 20 client.14988 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3fccb30 0x55a7f3fcb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 4 client.14988 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.536+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 --> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] -- command(tid 0: {"prefix": "status"}) -- 0x55a7f3e2d0e0 con 0x55a7f3fccb30 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f566affd640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3fccb30 0x55a7f3fcb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f566affd640 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3fccb30 0x55a7f3fcb840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdc14ef8654d60e0 server_cookie=e53fcf836ba603cd in_seq=0 out_seq=0 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f564affd640 10 client.14988 ms_handle_connect on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f564affd640 1 -- 172.21.15.31:0/364876382 <== mds.3 v2:172.21.15.102:6836/4082772592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7f3e2d0e0 con 0x55a7f3fccb30 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f564affd640 10 client.14988 handle_command_reply: tid=0 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 client.14988 shutdown 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 msgr2=0x55a7f3f63820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f566009bb70 tx=0x7f56600cd090 comp rx=0 tx=0).stop 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 shutdown_connections 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55a7f3fccb30 0x55a7f3fcb840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7f3f4dbb0 0x55a7f3fc6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a7f3f432f0 0x55a7f3f63820 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 --2- 172.21.15.31:0/364876382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7f3f429d0 0x55a7f3f632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f564affd640 0 client.14988 ms_handle_reset on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.537+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 >> 172.21.15.31:0/364876382 conn(0x55a7f3f3ef60 msgr2=0x55a7f3f3e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 shutdown_connections 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f5673d80740 1 -- 172.21.15.31:0/364876382 wait complete. 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f5673d80740 20 client.14988 trim_cache size 0 max 16384 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-09-15T23:59:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24457, 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 69.256949284000001, 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 104.658883467 2024-09-15T23:59:31.545 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 msgr2=0x7f56540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 0x7f56540a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664009260 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 msgr2=0x7f566c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f566000d950 tx=0x7f566000de10 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 shutdown_connections 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f56540a3350 0x7f56540a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f566c10e4f0 0x7f566c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c1012f0 0x7f566c19ab30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 --2- 172.21.15.31:0/1027007297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f566c100940 0x7f566c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 >> 172.21.15.31:0/1027007297 conn(0x7f566c0fc790 msgr2=0x7f566c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:31.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.538+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 shutdown_connections 2024-09-15T23:59:31.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:31.539+0000 7f56719a7640 1 -- 172.21.15.31:0/1027007297 wait complete. 2024-09-15T23:59:31.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:31 smithi081 ceph-mon[35060]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:32.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T23:59:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- 172.21.15.31:0/859138136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4101450 msgr2=0x7f43c4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 --2- 172.21.15.31:0/859138136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4101450 0x7f43c4105920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402a470 comp rx=0 tx=0).stop 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- 172.21.15.31:0/859138136 shutdown_connections 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 --2- 172.21.15.31:0/859138136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 0x7f43c41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 --2- 172.21.15.31:0/859138136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4101450 0x7f43c4105920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 --2- 172.21.15.31:0/859138136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4100b30 0x7f43c4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- 172.21.15.31:0/859138136 >> 172.21.15.31:0/859138136 conn(0x7f43c40fc7a0 msgr2=0x7f43c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- 172.21.15.31:0/859138136 shutdown_connections 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- 172.21.15.31:0/859138136 wait complete. 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 Processor -- start 2024-09-15T23:59:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.962+0000 7f43cc145640 1 -- start start 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4101450 0x7f43c41a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 0x7f43c419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41183f0 con 0x7f43c4100b30 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4118270 con 0x7f43c4105f40 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43cc145640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4118570 con 0x7f43c4101450 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52042/0 (socket says 172.21.15.31:52042) 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 -- 172.21.15.31:0/2661700956 learned_addr learned my addr 172.21.15.31:0/2661700956 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 0x7f43c419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c96e0640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4101450 0x7f43c41a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 -- 172.21.15.31:0/2661700956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4101450 msgr2=0x7f43c41a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4101450 0x7f43c41a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 -- 172.21.15.31:0/2661700956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 msgr2=0x7f43c419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 0x7f43c419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 -- 172.21.15.31:0/2661700956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419ef90 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c920 tx=0x7f43c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.963+0000 7f43baffd640 1 -- 172.21.15.31:0/2661700956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0007c70 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.964+0000 7f43baffd640 1 -- 172.21.15.31:0/2661700956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43c0007e10 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.964+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419f280 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.964+0000 7f43baffd640 1 -- 172.21.15.31:0/2661700956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018820 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.964+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41ab4d0 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.966+0000 7f43baffd640 1 -- 172.21.15.31:0/2661700956 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f43c00189c0 con 0x7f43c4100b30 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.966+0000 7f43ce51e740 1 Processor -- start 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.966+0000 7f43ce51e740 1 -- start start 2024-09-15T23:59:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.966+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d54a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.966+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ce51e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b3d540b090 con 0x55b3d54afbb0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ce51e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3d53b2a70 con 0x55b3d54a49d0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ce51e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b3d5407050 con 0x55b3d54a52f0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43baffd640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 0x7f43b00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52054/0 (socket says 172.21.15.31:52054) 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 -- 172.21.15.31:0/832800149 learned_addr learned my addr 172.21.15.31:0/832800149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 0x7f43b00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d54a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43baffd640 1 -- 172.21.15.31:0/2661700956 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f43c00cf910 con 0x7f43c4100b30 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 -- 172.21.15.31:0/832800149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 msgr2=0x55b3d54af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 -- 172.21.15.31:0/832800149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 msgr2=0x55b3d54a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d54a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 -- 172.21.15.31:0/832800149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d53bf130 con 0x55b3d54afbb0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 0x7f43b00a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009fd0 tx=0x7f43bc009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.967+0000 7f43c96e0640 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b4032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a97e9f9aca54fab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43affff640 1 -- 172.21.15.31:0/832800149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4040070 con 0x55b3d54afbb0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43affff640 1 -- 172.21.15.31:0/832800149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b4032e20 con 0x55b3d54afbb0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43affff640 1 -- 172.21.15.31:0/832800149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4035040 con 0x55b3d54afbb0 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 -- 172.21.15.31:0/832800149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 msgr2=0x55b3d54b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b4032cb0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 -- 172.21.15.31:0/832800149 shutdown_connections 2024-09-15T23:59:32.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54b1fa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 --2- 172.21.15.31:0/832800149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d54a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 -- 172.21.15.31:0/832800149 >> 172.21.15.31:0/832800149 conn(0x55b3d54a0f60 msgr2=0x55b3d54a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 -- 172.21.15.31:0/832800149 shutdown_connections 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.968+0000 7f43ce51e740 1 -- 172.21.15.31:0/832800149 wait complete. 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 -- start start 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 Processor -- start 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54b2780 con 0x55b3d54afbb0 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3d53c5f20 con 0x55b3d54a52f0 2024-09-15T23:59:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54b2900 con 0x55b3d54a49d0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.969+0000 7f43ce51e740 10 client.0 _ll_register_callbacks cb 0x55b3d538f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52070/0 (socket says 172.21.15.31:52070) 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 -- 172.21.15.31:0/329877886 learned_addr learned my addr 172.21.15.31:0/329877886 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43c96e0640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 -- 172.21.15.31:0/329877886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 msgr2=0x55b3d54af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 -- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 msgr2=0x55b3d54ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 -- 172.21.15.31:0/329877886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d517c2b0 con 0x55b3d54afbb0 2024-09-15T23:59:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f43bc010df0 tx=0x7f43bc009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a791ddaa1626bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc02a410 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f43bc02a5b0 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b4032e20 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43bc02a7b0 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ce51e740 10 client.15000 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.970+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3d54be070 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b4032e20 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43bc01f070 con 0x55b3d54afbb0 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 10 client.15000 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 20 client.15000 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d54c0700 0x55b3d54c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 4 client.15000 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 --> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3d5418600 con 0x55b3d54c0700 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d54c0700 0x55b3d54c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d54c0700 0x55b3d54c1b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=989c708caad317d8 server_cookie=dd5b54a538ba9be3 in_seq=0 out_seq=0 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.971+0000 7f43ad7fa640 10 client.15000 ms_handle_connect on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.972+0000 7f43ad7fa640 1 -- 172.21.15.31:0/329877886 <== mds.2 v2:172.21.15.81:6836/3184710981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3d5418600 con 0x55b3d54c0700 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.972+0000 7f43ad7fa640 10 client.15000 handle_command_reply: tid=0 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.972+0000 7f43ce51e740 1 client.15000 shutdown 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.972+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 msgr2=0x55b3d54accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.972+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f43bc010df0 tx=0x7f43bc009600 comp rx=0 tx=0).stop 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 shutdown_connections 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d54c0700 0x55b3d54c1b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54accb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ad7fa640 0 client.15000 ms_handle_reset on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 --2- 172.21.15.31:0/329877886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 >> 172.21.15.31:0/329877886 conn(0x55b3d54a0f60 msgr2=0x55b3d54a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 shutdown_connections 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 1 -- 172.21.15.31:0/329877886 wait complete. 2024-09-15T23:59:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.973+0000 7f43ce51e740 20 client.15000 trim_cache size 0 max 16384 2024-09-15T23:59:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 Processor -- start 2024-09-15T23:59:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 -- start start 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d549bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54bfeb0 con 0x55b3d54afbb0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54c0580 con 0x55b3d54a49d0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ce51e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54c0c50 con 0x55b3d54a52f0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52084/0 (socket says 172.21.15.31:52084) 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3607588105 learned_addr learned my addr 172.21.15.31:0/3607588105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43c96e0640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d549bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3607588105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 msgr2=0x55b3d54ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3607588105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 msgr2=0x55b3d549bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d549bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3607588105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d54af040 con 0x55b3d54afbb0 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d549bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.991+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0092f0 tx=0x7f43bc0127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc042bf9fbb978e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43af7fe640 1 -- 172.21.15.31:0/3607588105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc009920 con 0x55b3d54afbb0 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43af7fe640 1 -- 172.21.15.31:0/3607588105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43bc012810 con 0x55b3d54afbb0 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43af7fe640 1 -- 172.21.15.31:0/3607588105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004c60 con 0x55b3d54afbb0 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 -- 172.21.15.31:0/3607588105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 msgr2=0x55b3d54ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0092f0 tx=0x7f43bc0127a0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 -- 172.21.15.31:0/3607588105 shutdown_connections 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d54ab580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a52f0 0x55b3d54ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3607588105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a49d0 0x55b3d549bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.992+0000 7f43ce51e740 1 -- 172.21.15.31:0/3607588105 >> 172.21.15.31:0/3607588105 conn(0x55b3d54a0f60 msgr2=0x55b3d54a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- 172.21.15.31:0/3607588105 shutdown_connections 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- 172.21.15.31:0/3607588105 wait complete. 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- start start 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 Processor -- start 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54c19f0 con 0x55b3d54afbb0 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54c1320 con 0x55b3d54a52f0 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.993+0000 7f43ce51e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b3d54c20c0 con 0x55b3d54a49d0 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ce51e740 10 client.0 _ll_register_callbacks cb 0x7f43bc012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43c9ee1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52098/0 (socket says 172.21.15.31:52098) 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3387446248 learned_addr learned my addr 172.21.15.31:0/3387446248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43c96e0640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3387446248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 msgr2=0x55b3d54c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 msgr2=0x55b3d54c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 -- 172.21.15.31:0/3387446248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d55234b0 con 0x55b3d54afbb0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ca6e2640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f43bc010fa0 tx=0x7f43bc01ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e55b2e29f58b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc020810 con 0x55b3d54afbb0 2024-09-15T23:59:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f43bc004a00 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.994+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc012810 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43bc009920 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 10 client.15006 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3d552a830 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43bc02ae00 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43bc00f040 con 0x55b3d54afbb0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 10 client.15006 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 10 client.15006 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 20 client.15006 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d552efe0 0x55b3d552dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 4 client.15006 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 --> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] -- command(tid 0: {"prefix": "status"}) -- 0x7f43bc012810 con 0x55b3d552efe0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.995+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d552efe0 0x55b3d552dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:33.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43c9ee1640 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d552efe0 0x55b3d552dcf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44e062afbb5da27a server_cookie=9c70f5800649e303 in_seq=0 out_seq=0 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ad7fa640 10 client.15006 ms_handle_connect on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ad7fa640 1 -- 172.21.15.31:0/3387446248 <== mds.2 v2:172.21.15.81:6836/3184710981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x7f43bc012810 con 0x55b3d552efe0 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ad7fa640 10 client.15006 handle_command_reply: tid=0 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 client.15006 shutdown 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 msgr2=0x55b3d5522e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f43bc010fa0 tx=0x7f43bc01ec80 comp rx=0 tx=0).stop 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 shutdown_connections 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x55b3d552efe0 0x55b3d552dcf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b3d54afbb0 0x55b3d5522e40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3d54a52f0 0x55b3d54c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ad7fa640 0 client.15006 ms_handle_reset on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 --2- 172.21.15.31:0/3387446248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3d54a49d0 0x55b3d54c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 >> 172.21.15.31:0/3387446248 conn(0x55b3d54a0f60 msgr2=0x55b3d54aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 shutdown_connections 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.996+0000 7f43ce51e740 1 -- 172.21.15.31:0/3387446248 wait complete. 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43ce51e740 20 client.15006 trim_cache size 0 max 16384 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24461, 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:33.009 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 71.722311833000006, 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 103.41018941 2024-09-15T23:59:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 msgr2=0x7f43b00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 0x7f43b00a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009fd0 tx=0x7f43bc009d80 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 msgr2=0x7f43c41a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c920 tx=0x7f43c000cde0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 shutdown_connections 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f43b00a3350 0x7f43b00a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c4105f40 0x7f43c419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43c4101450 0x7f43c41a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 --2- 172.21.15.31:0/2661700956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c4100b30 0x7f43c41a4670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 >> 172.21.15.31:0/2661700956 conn(0x7f43c40fc7a0 msgr2=0x7f43c410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 shutdown_connections 2024-09-15T23:59:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:32.997+0000 7f43cc145640 1 -- 172.21.15.31:0/2661700956 wait complete. 2024-09-15T23:59:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:33 smithi031 ceph-mon[31203]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:33 smithi102 ceph-mon[34458]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:33 smithi081 ceph-mon[35060]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:33.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T23:59:33.930 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 -- 172.21.15.31:0/956117421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c101380 msgr2=0x7f9e7c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 --2- 172.21.15.31:0/956117421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c101380 0x7f9e7c10dfb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009900 tx=0x7f9e6c02a410 comp rx=0 tx=0).stop 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 -- 172.21.15.31:0/956117421 shutdown_connections 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 --2- 172.21.15.31:0/956117421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c10e4f0 0x7f9e7c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 --2- 172.21.15.31:0/956117421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c101380 0x7f9e7c10dfb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 --2- 172.21.15.31:0/956117421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 -- 172.21.15.31:0/956117421 >> 172.21.15.31:0/956117421 conn(0x7f9e7c0fc7d0 msgr2=0x7f9e7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 -- 172.21.15.31:0/956117421 shutdown_connections 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.386+0000 7f9e837f6640 1 -- 172.21.15.31:0/956117421 wait complete. 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 Processor -- start 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 -- start start 2024-09-15T23:59:34.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c101380 0x7f9e7c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c10e4f0 0x7f9e7c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c114110 con 0x7f9e7c1009d0 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c113f90 con 0x7f9e7c10e4f0 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e837f6640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c114290 con 0x7f9e7c101380 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52110/0 (socket says 172.21.15.31:52110) 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 -- 172.21.15.31:0/1421101546 learned_addr learned my addr 172.21.15.31:0/1421101546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e80d91640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c101380 0x7f9e7c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81d93640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c10e4f0 0x7f9e7c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 -- 172.21.15.31:0/1421101546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c101380 msgr2=0x7f9e7c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c101380 0x7f9e7c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 -- 172.21.15.31:0/1421101546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c10e4f0 msgr2=0x7f9e7c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c10e4f0 0x7f9e7c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.387+0000 7f9e81592640 1 -- 172.21.15.31:0/1421101546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c19f5a0 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e81592640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800cce0 tx=0x7f9e7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e727fc640 1 -- 172.21.15.31:0/1421101546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800f650 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e727fc640 1 -- 172.21.15.31:0/1421101546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e7800fca0 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e727fc640 1 -- 172.21.15.31:0/1421101546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78018b90 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c19f890 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.388+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a7170 con 0x7f9e7c1009d0 2024-09-15T23:59:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.390+0000 7f9e727fc640 1 -- 172.21.15.31:0/1421101546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9e7800f7f0 con 0x7f9e7c1009d0 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 Processor -- start 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 -- start start 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563c5f40e090 con 0x563c5f4a79d0 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563c5f3b5a70 con 0x563c5f4b2bb0 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c5f40a050 con 0x563c5f4a82f0 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52122/0 (socket says 172.21.15.31:52122) 2024-09-15T23:59:34.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 -- 172.21.15.31:0/2644058790 learned_addr learned my addr 172.21.15.31:0/2644058790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81592640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e80d91640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e727fc640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 0x7f9e680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 -- 172.21.15.31:0/2644058790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 msgr2=0x563c5f4b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 -- 172.21.15.31:0/2644058790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 msgr2=0x563c5f4b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e81d93640 1 -- 172.21.15.31:0/2644058790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5f3c2130 con 0x563c5f4a79d0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.391+0000 7f9e80d91640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e727fc640 1 -- 172.21.15.31:0/1421101546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9e780cfe30 con 0x7f9e7c1009d0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e81d93640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 0x7f9e680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b600 tx=0x7f9e7400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a6e5c695d354b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e677fe640 1 -- 172.21.15.31:0/2644058790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74003ce0 con 0x563c5f4a79d0 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e677fe640 1 -- 172.21.15.31:0/2644058790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e74015070 con 0x563c5f4a79d0 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.392+0000 7f9e677fe640 1 -- 172.21.15.31:0/2644058790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7401bb20 con 0x563c5f4a79d0 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2644058790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 msgr2=0x563c5f4a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e81d93640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 0x7f9e680a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9e7401c000 tx=0x7f9e74010810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b600 tx=0x7f9e7400bac0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2644058790 shutdown_connections 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2644058790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a79d0 0x563c5f4a7db0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2644058790 >> 172.21.15.31:0/2644058790 conn(0x563c5f4a3f60 msgr2=0x563c5f4a4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2644058790 shutdown_connections 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2644058790 wait complete. 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 -- start start 2024-09-15T23:59:34.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.393+0000 7f9e85bcf740 1 Processor -- start 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4b5780 con 0x563c5f4a82f0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563c5f3c8f20 con 0x563c5f4b2bb0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4b5900 con 0x563c5f4a79d0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e85bcf740 10 client.0 _ll_register_callbacks cb 0x563c5f392030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52132/0 (socket says 172.21.15.31:52132) 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 -- 172.21.15.31:0/782771701 learned_addr learned my addr 172.21.15.31:0/782771701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e81d93640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e81592640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 -- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 msgr2=0x563c5f4b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 -- 172.21.15.31:0/782771701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 msgr2=0x563c5f4afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 -- 172.21.15.31:0/782771701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5f17f2b0 con 0x563c5f4a82f0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e81d93640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e80d91640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009900 tx=0x7f9e6c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9505fdc7249a94ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.394+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c035040 con 0x563c5f4a82f0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9e6c004240 con 0x563c5f4a82f0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e74015070 con 0x563c5f4a82f0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e6c0318c0 con 0x563c5f4a82f0 2024-09-15T23:59:34.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 10 client.15018 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c5f4c1030 con 0x563c5f4a82f0 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e74015070 con 0x563c5f4a82f0 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e6c0049b0 con 0x563c5f4a82f0 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 10 client.15018 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 20 client.15018 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f4c36a0 0x563c5f4c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 4 client.15018 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.395+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 --> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c5f41b600 con 0x563c5f4c36a0 2024-09-15T23:59:34.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.396+0000 7f9e81592640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f4c36a0 0x563c5f4c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.396+0000 7f9e64ff9640 10 client.15018 ms_handle_connect on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:34.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.396+0000 7f9e81592640 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f4c36a0 0x563c5f4c4b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c15e483324364ea server_cookie=6a0caf7f257a6cbc in_seq=0 out_seq=0 2024-09-15T23:59:34.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e64ff9640 1 -- 172.21.15.31:0/782771701 <== mds.1 v2:172.21.15.102:6838/3686122257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c5f41b600 con 0x563c5f4c36a0 2024-09-15T23:59:34.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e64ff9640 10 client.15018 handle_command_reply: tid=0 2024-09-15T23:59:34.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 client.15018 shutdown 2024-09-15T23:59:34.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 msgr2=0x563c5f4af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009900 tx=0x7f9e6c0316d0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 shutdown_connections 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f4c36a0 0x563c5f4c4b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4b2bb0 0x563c5f4afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4a82f0 0x563c5f4af720 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/782771701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 >> 172.21.15.31:0/782771701 conn(0x563c5f4a3f60 msgr2=0x563c5f4a9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e64ff9640 0 client.15018 ms_handle_reset on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 shutdown_connections 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 1 -- 172.21.15.31:0/782771701 wait complete. 2024-09-15T23:59:34.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.397+0000 7f9e85bcf740 20 client.15018 trim_cache size 0 max 16384 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 Processor -- start 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 -- start start 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a79d0 0x563c5f4c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c2e50 con 0x563c5f4b2bb0 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c3520 con 0x563c5f4a79d0 2024-09-15T23:59:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c3bf0 con 0x563c5f4a82f0 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.415+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52136/0 (socket says 172.21.15.31:52136) 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 -- 172.21.15.31:0/155570697 learned_addr learned my addr 172.21.15.31:0/155570697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81592640 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a79d0 0x563c5f4c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e80d91640 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 -- 172.21.15.31:0/155570697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 msgr2=0x563c5f4b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 -- 172.21.15.31:0/155570697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a79d0 msgr2=0x563c5f4c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a79d0 0x563c5f4c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 -- 172.21.15.31:0/155570697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5f4b5890 con 0x563c5f4b2bb0 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e81d93640 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e74003670 tx=0x7f9e74026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e506891e0c11b9bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e66ffd640 1 -- 172.21.15.31:0/155570697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7402d050 con 0x563c5f4b2bb0 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e66ffd640 1 -- 172.21.15.31:0/155570697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e74037070 con 0x563c5f4b2bb0 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.416+0000 7f9e66ffd640 1 -- 172.21.15.31:0/155570697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74005040 con 0x563c5f4b2bb0 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- 172.21.15.31:0/155570697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 msgr2=0x563c5f4ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e74003670 tx=0x7f9e74026000 comp rx=0 tx=0).stop 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- 172.21.15.31:0/155570697 shutdown_connections 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4ccdf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a82f0 0x563c5f4b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/155570697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a79d0 0x563c5f4c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- 172.21.15.31:0/155570697 >> 172.21.15.31:0/155570697 conn(0x563c5f4a3f60 msgr2=0x563c5f4adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- 172.21.15.31:0/155570697 shutdown_connections 2024-09-15T23:59:34.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- 172.21.15.31:0/155570697 wait complete. 2024-09-15T23:59:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 -- start start 2024-09-15T23:59:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.417+0000 7f9e85bcf740 1 Processor -- start 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a82f0 0x563c5f4c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c4990 con 0x563c5f4b2bb0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c42c0 con 0x563c5f4a82f0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c5060 con 0x563c5f4a79d0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e85bcf740 10 client.0 _ll_register_callbacks cb 0x7f9e6c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e80d91640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a82f0 0x563c5f4c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52150/0 (socket says 172.21.15.31:52150) 2024-09-15T23:59:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 -- 172.21.15.31:0/2307981576 learned_addr learned my addr 172.21.15.31:0/2307981576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81592640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 -- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 msgr2=0x563c5f4c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 -- 172.21.15.31:0/2307981576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a82f0 msgr2=0x563c5f4c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a82f0 0x563c5f4c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 -- 172.21.15.31:0/2307981576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5f52bea0 con 0x563c5f4b2bb0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81592640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e81d93640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005f00 tx=0x7f9e74011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88de107b32a20c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.418+0000 7f9e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74040070 con 0x563c5f4b2bb0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9e74003940 con 0x563c5f4b2bb0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e74037070 con 0x563c5f4b2bb0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e7402d020 con 0x563c5f4b2bb0 2024-09-15T23:59:34.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 10 client.15024 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c5f52d1a0 con 0x563c5f4b2bb0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e74037070 con 0x563c5f4b2bb0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e7401b5c0 con 0x563c5f4b2bb0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 10 client.15024 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 10 client.15024 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 20 client.15024 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f5316e0 0x563c5f5303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 4 client.15024 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.419+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 --> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] -- command(tid 0: {"prefix": "status"}) -- 0x563c5f415b10 con 0x563c5f5316e0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e81592640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f5316e0 0x563c5f5303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e64ff9640 10 client.15024 ms_handle_connect on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e81592640 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f5316e0 0x563c5f5303f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1972f9893117e86 server_cookie=405ff7bcb7845a9e in_seq=0 out_seq=0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e64ff9640 1 -- 172.21.15.31:0/2307981576 <== mds.1 v2:172.21.15.102:6838/3686122257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x563c5f415b10 con 0x563c5f5316e0 2024-09-15T23:59:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e64ff9640 10 client.15024 handle_command_reply: tid=0 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 client.15024 shutdown 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 msgr2=0x563c5f4c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005f00 tx=0x7f9e74011f50 comp rx=0 tx=0).stop 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 shutdown_connections 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x563c5f5316e0 0x563c5f5303f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563c5f4b2bb0 0x563c5f4c8650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5f4a82f0 0x563c5f4c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 --2- 172.21.15.31:0/2307981576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c5f4a79d0 0x563c5f4c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 >> 172.21.15.31:0/2307981576 conn(0x563c5f4a3f60 msgr2=0x563c5f4ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e64ff9640 0 client.15024 ms_handle_reset on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 shutdown_connections 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.420+0000 7f9e85bcf740 1 -- 172.21.15.31:0/2307981576 wait complete. 2024-09-15T23:59:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e85bcf740 20 client.15024 trim_cache size 0 max 16384 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24478, 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 74.15147236, 2024-09-15T23:59:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 102.210463096 2024-09-15T23:59:34.429 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 msgr2=0x7f9e680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 0x7f9e680a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9e7401c000 tx=0x7f9e74010810 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 msgr2=0x7f9e7c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800cce0 tx=0x7f9e7800a620 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 shutdown_connections 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f9e680a32d0 0x7f9e680a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e7c10e4f0 0x7f9e7c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e7c101380 0x7f9e7c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 --2- 172.21.15.31:0/1421101546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e7c1009d0 0x7f9e7c19a5f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 >> 172.21.15.31:0/1421101546 conn(0x7f9e7c0fc7d0 msgr2=0x7f9e7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:34.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 shutdown_connections 2024-09-15T23:59:34.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:34.421+0000 7f9e837f6640 1 -- 172.21.15.31:0/1421101546 wait complete. 2024-09-15T23:59:35.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T23:59:35.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:35 smithi102 ceph-mon[34458]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:35 smithi031 ceph-mon[31203]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:35.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:35 smithi081 ceph-mon[35060]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-15T23:59:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 -- 172.21.15.31:0/1490065569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 msgr2=0x7f7e3c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 --2- 172.21.15.31:0/1490065569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c100ef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009930 tx=0x7f7e2402a420 comp rx=0 tx=0).stop 2024-09-15T23:59:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 -- 172.21.15.31:0/1490065569 shutdown_connections 2024-09-15T23:59:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 --2- 172.21.15.31:0/1490065569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 0x7f7e3c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 --2- 172.21.15.31:0/1490065569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c100ef0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 --2- 172.21.15.31:0/1490065569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 0x7f7e3c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 -- 172.21.15.31:0/1490065569 >> 172.21.15.31:0/1490065569 conn(0x7f7e3c0fc7d0 msgr2=0x7f7e3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 -- 172.21.15.31:0/1490065569 shutdown_connections 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.964+0000 7f7e42352640 1 -- 172.21.15.31:0/1490065569 wait complete. 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 Processor -- start 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 -- start start 2024-09-15T23:59:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 0x7f7e3c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 0x7f7e3c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110d40 con 0x7f7e3c100a90 2024-09-15T23:59:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110bc0 con 0x7f7e3c101430 2024-09-15T23:59:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e42352640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c110ec0 con 0x7f7e3c106a90 2024-09-15T23:59:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52178/0 (socket says 172.21.15.31:52178) 2024-09-15T23:59:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1192975737 learned_addr learned my addr 172.21.15.31:0/1192975737 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 0x7f7e3c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e33fff640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 0x7f7e3c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1192975737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 msgr2=0x7f7e3c1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 0x7f7e3c1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1192975737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 msgr2=0x7f7e3c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 0x7f7e3c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1192975737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c1a47d0 con 0x7f7e3c100a90 2024-09-15T23:59:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.965+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00e9f0 tx=0x7f7e2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.966+0000 7f7e39ffb640 1 -- 172.21.15.31:0/1192975737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00cd80 con 0x7f7e3c100a90 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.966+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c1a49a0 con 0x7f7e3c100a90 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.966+0000 7f7e39ffb640 1 -- 172.21.15.31:0/1192975737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2c0054d0 con 0x7f7e3c100a90 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.966+0000 7f7e39ffb640 1 -- 172.21.15.31:0/1192975737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c010490 con 0x7f7e3c100a90 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.966+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a4eb0 con 0x7f7e3c100a90 2024-09-15T23:59:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e39ffb640 1 -- 172.21.15.31:0/1192975737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7e2c005020 con 0x7f7e3c100a90 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 Processor -- start 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 -- start start 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560b5b314090 con 0x560b5b3ad9d0 2024-09-15T23:59:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560b5b2bba70 con 0x560b5b3ae2f0 2024-09-15T23:59:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e4472b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b5b310050 con 0x560b5b3b8bb0 2024-09-15T23:59:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.968+0000 7f7e408ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52182/0 (socket says 172.21.15.31:52182) 2024-09-15T23:59:35.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 -- 172.21.15.31:0/1763639812 learned_addr learned my addr 172.21.15.31:0/1763639812 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e39ffb640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 0x7f7e280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e33fff640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 -- 172.21.15.31:0/1763639812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 msgr2=0x560b5b3bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:35.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 -- 172.21.15.31:0/1763639812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 msgr2=0x560b5b3b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 -- 172.21.15.31:0/1763639812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5b2c8130 con 0x560b5b3ad9d0 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 0x7f7e280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e39ffb640 1 -- 172.21.15.31:0/1192975737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7e2c0cfd20 con 0x7f7e3c100a90 2024-09-15T23:59:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7e340103f0 tx=0x7f7e340108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc60b56490f0745 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e327fc640 1 -- 172.21.15.31:0/1763639812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3401c900 con 0x560b5b3ad9d0 2024-09-15T23:59:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e327fc640 1 -- 172.21.15.31:0/1763639812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e3401b3d0 con 0x560b5b3ad9d0 2024-09-15T23:59:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.969+0000 7f7e327fc640 1 -- 172.21.15.31:0/1763639812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34026680 con 0x560b5b3ad9d0 2024-09-15T23:59:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 -- 172.21.15.31:0/1763639812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 msgr2=0x560b5b3addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e408ef640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 0x7f7e280a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e34020000 tx=0x7f7e3401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7e340103f0 tx=0x7f7e340108b0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 -- 172.21.15.31:0/1763639812 shutdown_connections 2024-09-15T23:59:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1763639812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3addb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 -- 172.21.15.31:0/1763639812 >> 172.21.15.31:0/1763639812 conn(0x560b5b3a9f60 msgr2=0x560b5b3aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 -- 172.21.15.31:0/1763639812 shutdown_connections 2024-09-15T23:59:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.970+0000 7f7e4472b740 1 -- 172.21.15.31:0/1763639812 wait complete. 2024-09-15T23:59:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 -- start start 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 Processor -- start 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3bb780 con 0x560b5b3ae2f0 2024-09-15T23:59:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560b5b2cef20 con 0x560b5b3ad9d0 2024-09-15T23:59:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3bb900 con 0x560b5b3b8bb0 2024-09-15T23:59:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e4472b740 10 client.0 _ll_register_callbacks cb 0x560b5b298030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52192/0 (socket says 172.21.15.31:52192) 2024-09-15T23:59:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e33fff640 1 -- 172.21.15.31:0/4018021771 learned_addr learned my addr 172.21.15.31:0/4018021771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.971+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e408ef640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 -- 172.21.15.31:0/4018021771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 msgr2=0x560b5b3b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 -- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ad9d0 msgr2=0x560b5b3b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 -- 172.21.15.31:0/4018021771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5b0852b0 con 0x560b5b3ae2f0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e408ef640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e33fff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e24005ab0 tx=0x7f7e24031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cade61c17051a704 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2403f070 con 0x560b5b3ae2f0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7e24004450 con 0x560b5b3ae2f0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3401b3d0 con 0x560b5b3ae2f0 2024-09-15T23:59:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e24035070 con 0x560b5b3ae2f0 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e4472b740 10 client.15036 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.972+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b5b3c70a0 con 0x560b5b3ae2f0 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e24048030 con 0x560b5b3ae2f0 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e2403b430 con 0x560b5b3ae2f0 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 20 client.15036 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b3cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 4 client.15036 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 --> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5b321600 con 0x560b5b3c9720 2024-09-15T23:59:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.973+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b3cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.974+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b3cab90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9015139e6f597d9 server_cookie=2bd24cecc827f21c in_seq=0 out_seq=0 2024-09-15T23:59:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.974+0000 7f7e1b7fe640 10 client.15036 ms_handle_connect on v2:172.21.15.81:6838/471643993 2024-09-15T23:59:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.974+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/4018021771 <== mds.0 v2:172.21.15.81:6838/471643993 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b5b321600 con 0x560b5b3c9720 2024-09-15T23:59:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.974+0000 7f7e1b7fe640 10 client.15036 handle_command_reply: tid=0 2024-09-15T23:59:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.974+0000 7f7e4472b740 1 client.15036 shutdown 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 msgr2=0x560b5b3b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e24005ab0 tx=0x7f7e24031750 comp rx=0 tx=0).stop 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 shutdown_connections 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b3cab90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b5760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e1b7fe640 0 client.15036 ms_handle_reset on v2:172.21.15.81:6838/471643993 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 --2- 172.21.15.31:0/4018021771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 >> 172.21.15.31:0/4018021771 conn(0x560b5b3a9f60 msgr2=0x560b5b3afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 shutdown_connections 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 1 -- 172.21.15.31:0/4018021771 wait complete. 2024-09-15T23:59:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.975+0000 7f7e4472b740 20 client.15036 trim_cache size 0 max 16384 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 Processor -- start 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 -- start start 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3c8ed0 con 0x560b5b3ad9d0 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3c95a0 con 0x560b5b3b8bb0 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e4472b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3c9c70 con 0x560b5b3ae2f0 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52194/0 (socket says 172.21.15.31:52194) 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1660923688 learned_addr learned my addr 172.21.15.31:0/1660923688 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1660923688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ae2f0 msgr2=0x560b5b3b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.993+0000 7f7e33fff640 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1660923688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 msgr2=0x560b5b3d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e3bfff640 1 -- 172.21.15.31:0/1660923688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5b3b4710 con 0x560b5b3ad9d0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0bafd0 tx=0x7f7e2c0d0070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33507f25067d68d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e31ffb640 1 -- 172.21.15.31:0/1660923688 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c005680 con 0x560b5b3ad9d0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e31ffb640 1 -- 172.21.15.31:0/1660923688 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2c09ca90 con 0x560b5b3ad9d0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e31ffb640 1 -- 172.21.15.31:0/1660923688 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0046c0 con 0x560b5b3ad9d0 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e4472b740 1 -- 172.21.15.31:0/1660923688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 msgr2=0x560b5b3b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.994+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0bafd0 tx=0x7f7e2c0d0070 comp rx=0 tx=0).stop 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 -- 172.21.15.31:0/1660923688 shutdown_connections 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 --2- 172.21.15.31:0/1660923688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ad9d0 0x560b5b3b4030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 -- 172.21.15.31:0/1660923688 >> 172.21.15.31:0/1660923688 conn(0x560b5b3a9f60 msgr2=0x560b5b3b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 -- 172.21.15.31:0/1660923688 shutdown_connections 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 -- 172.21.15.31:0/1660923688 wait complete. 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 -- start start 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 Processor -- start 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ad9d0 0x560b5b3d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.995+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3caa10 con 0x560b5b3ae2f0 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3ca340 con 0x560b5b3b8bb0 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b5b3cb0e0 con 0x560b5b3ad9d0 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 10 client.0 _ll_register_callbacks cb 0x7f7e2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52208/0 (socket says 172.21.15.31:52208) 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 -- 172.21.15.31:0/2412409244 learned_addr learned my addr 172.21.15.31:0/2412409244 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ad9d0 0x560b5b3d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e408ef640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 -- 172.21.15.31:0/2412409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ad9d0 msgr2=0x560b5b3d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ad9d0 0x560b5b3d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 -- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 msgr2=0x560b5b3b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 -- 172.21.15.31:0/2412409244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5b3d11a0 con 0x560b5b3ae2f0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e33fff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7e2403ba40 tx=0x7f7e2403b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a11ee9f964cf1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2403f040 con 0x560b5b3ae2f0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e2c09ca90 con 0x560b5b3ae2f0 2024-09-15T23:59:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.996+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7e2403bac0 con 0x560b5b3ae2f0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e2404b020 con 0x560b5b3ae2f0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 10 client.15042 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b5b434010 con 0x560b5b3ae2f0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2c09ca90 con 0x560b5b3ae2f0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e240046c0 con 0x560b5b3ae2f0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 10 client.15042 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 20 client.15042 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b4378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 4 client.15042 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 --> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] -- command(tid 0: {"prefix": "status"}) -- 0x560b5b31bb10 con 0x560b5b3c9720 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.997+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b4378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e3bfff640 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b4378e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbf370832710b30 server_cookie=5fd5e2d7a6e98357 in_seq=0 out_seq=0 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e1b7fe640 10 client.15042 ms_handle_connect on v2:172.21.15.81:6838/471643993 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e1b7fe640 1 -- 172.21.15.31:0/2412409244 <== mds.0 v2:172.21.15.81:6838/471643993 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560b5b31bb10 con 0x560b5b3c9720 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e1b7fe640 10 client.15042 handle_command_reply: tid=0 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 client.15042 shutdown 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 msgr2=0x560b5b3b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7e2403ba40 tx=0x7f7e2403b6c0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 shutdown_connections 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b5b3b8bb0 0x560b5b3b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560b5b3ae2f0 0x560b5b3b2650 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x560b5b3c9720 0x560b5b4378e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 --2- 172.21.15.31:0/2412409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b5b3ad9d0 0x560b5b3d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 >> 172.21.15.31:0/2412409244 conn(0x560b5b3a9f60 msgr2=0x560b5b3b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e1b7fe640 0 client.15042 ms_handle_reset on v2:172.21.15.81:6838/471643993 2024-09-15T23:59:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 shutdown_connections 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.998+0000 7f7e4472b740 1 -- 172.21.15.31:0/2412409244 wait complete. 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e4472b740 20 client.15042 trim_cache size 0 max 16384 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24491, 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 83.858250545999994, 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 94.646822893999996 2024-09-15T23:59:36.010 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 msgr2=0x7f7e280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 0x7f7e280a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e34020000 tx=0x7f7e3401d000 comp rx=0 tx=0).stop 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 msgr2=0x7f7e3c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00e9f0 tx=0x7f7e2c00edf0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 shutdown_connections 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7e280a32d0 0x7f7e280a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e3c106a90 0x7f7e3c1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e3c101430 0x7f7e3c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 --2- 172.21.15.31:0/1192975737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c100a90 0x7f7e3c199210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 >> 172.21.15.31:0/1192975737 conn(0x7f7e3c0fc7d0 msgr2=0x7f7e3c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 shutdown_connections 2024-09-15T23:59:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:35.999+0000 7f7e42352640 1 -- 172.21.15.31:0/1192975737 wait complete. 2024-09-15T23:59:36.711 INFO:tasks.cephadm:Setup complete, yielding 2024-09-15T23:59:36.711 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T23:59:36.720 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-15T23:59:36.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-15T23:59:36.980 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:37.351 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:37 smithi031 ceph-mon[31203]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:37 smithi102 ceph-mon[34458]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 -- 172.21.15.31:0/2751624374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100b10 msgr2=0x7f6900100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 --2- 172.21.15.31:0/2751624374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100b10 0x7f6900100f10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009900 tx=0x7f68e802a410 comp rx=0 tx=0).stop 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 -- 172.21.15.31:0/2751624374 shutdown_connections 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 --2- 172.21.15.31:0/2751624374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69001026d0 0x7f6900108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 --2- 172.21.15.31:0/2751624374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 0x7f6900102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 --2- 172.21.15.31:0/2751624374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100b10 0x7f6900100f10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 -- 172.21.15.31:0/2751624374 >> 172.21.15.31:0/2751624374 conn(0x7f69000fc2a0 msgr2=0x7f69000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.429+0000 7f6904813640 1 -- 172.21.15.31:0/2751624374 shutdown_connections 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 -- 172.21.15.31:0/2751624374 wait complete. 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 Processor -- start 2024-09-15T23:59:37.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 -- start start 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 0x7f690006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 0x7f690019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f690010c650 con 0x7f69001026d0 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f690010c4d0 con 0x7f6900101d10 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.430+0000 7f6904813640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f690010c7d0 con 0x7f6900100b10 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52230/0 (socket says 172.21.15.31:52230) 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 -- 172.21.15.31:0/209384176 learned_addr learned my addr 172.21.15.31:0/209384176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fdd9b640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 0x7f690019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fe59c640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 0x7f690006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 -- 172.21.15.31:0/209384176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 msgr2=0x7f690006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:37.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 0x7f690006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 -- 172.21.15.31:0/209384176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 msgr2=0x7f690019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 0x7f690019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 -- 172.21.15.31:0/209384176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690019fac0 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fe59c640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 0x7f690006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68fed9d640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68f800eb10 tx=0x7f68f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004b20 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68f8004cc0 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690019fd50 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8010690 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.431+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69001a0290 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.432+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6900108020 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.433+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f68f8005020 con 0x7f69001026d0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.434+0000 7f68f77fe640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 0x7f68ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.434+0000 7f68fe59c640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 0x7f68ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.434+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f68f80d0040 con 0x7f69001026d0 2024-09-15T23:59:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.435+0000 7f68fe59c640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 0x7f68ec0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000c00 tx=0x7f68e8032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.439+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68f809cdb0 con 0x7f69001026d0 2024-09-15T23:59:37.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:37 smithi081 ceph-mon[35060]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:37.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.629+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f6900062630 con 0x7f68ec0a32d0 2024-09-15T23:59:37.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.631+0000 7f68f77fe640 1 -- 172.21.15.31:0/209384176 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f6900062630 con 0x7f68ec0a32d0 2024-09-15T23:59:37.632 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-09-15T23:59:37.633 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-09-15T23:59:37.633 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 msgr2=0x7f68ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 0x7f68ec0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000c00 tx=0x7f68e8032c40 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 msgr2=0x7f690019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68f800eb10 tx=0x7f68f800c2d0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 shutdown_connections 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f68ec0a32d0 0x7f68ec0a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69001026d0 0x7f690019f1b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6900101d10 0x7f690019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 --2- 172.21.15.31:0/209384176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6900100b10 0x7f690006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 >> 172.21.15.31:0/209384176 conn(0x7f69000fc2a0 msgr2=0x7f69001051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 shutdown_connections 2024-09-15T23:59:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:37.633+0000 7f6904813640 1 -- 172.21.15.31:0/209384176 wait complete. 2024-09-15T23:59:38.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-15T23:59:38.521 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 -- 172.21.15.31:0/2871779766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc107900 msgr2=0x7fbdfc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 --2- 172.21.15.31:0/2871779766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc107900 0x7fbdfc109d50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009ca0 tx=0x7fbdf402a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 -- 172.21.15.31:0/2871779766 shutdown_connections 2024-09-15T23:59:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 --2- 172.21.15.31:0/2871779766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc107900 0x7fbdfc109d50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 --2- 172.21.15.31:0/2871779766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 --2- 172.21.15.31:0/2871779766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 -- 172.21.15.31:0/2871779766 >> 172.21.15.31:0/2871779766 conn(0x7fbdfc0fc2a0 msgr2=0x7fbdfc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 -- 172.21.15.31:0/2871779766 shutdown_connections 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.004+0000 7fbe026ad640 1 -- 172.21.15.31:0/2871779766 wait complete. 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 Processor -- start 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 -- start start 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc107900 0x7fbdfc19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc10d770 con 0x7fbdfc1026a0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc10d5f0 con 0x7fbdfc104fd0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.005+0000 7fbe026ad640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc10d8f0 con 0x7fbdfc107900 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52254/0 (socket says 172.21.15.31:52254) 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 -- 172.21.15.31:0/728396048 learned_addr learned my addr 172.21.15.31:0/728396048 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfb7fe640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbe00c4a640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc107900 0x7fbdfc19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 -- 172.21.15.31:0/728396048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc107900 msgr2=0x7fbdfc19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc107900 0x7fbdfc19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 -- 172.21.15.31:0/728396048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 msgr2=0x7fbdfc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 -- 172.21.15.31:0/728396048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc19fa60 con 0x7fbdfc1026a0 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfb7fe640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdfbfff640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c970 tx=0x7fbdec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec007bb0 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdec007d50 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc19fd50 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00f450 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.006+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc1a01d0 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.007+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdfc0fdd90 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.008+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbdec004ee0 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.009+0000 7fbdf97fa640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 0x7fbde40a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.009+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fbdec0cf9b0 con 0x7fbdfc1026a0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.009+0000 7fbdfb7fe640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 0x7fbde40a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.010+0000 7fbdfb7fe640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 0x7fbde40a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00058c0 tx=0x7fbdf0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.012+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdec014030 con 0x7fbdfc1026a0 2024-09-15T23:59:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.200+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbdfc062630 con 0x7fbde40a3380 2024-09-15T23:59:39.202 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:39 smithi031 ceph-mon[31203]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:39.202 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:39 smithi031 ceph-mon[31203]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.212+0000 7fbdf97fa640 1 -- 172.21.15.31:0/728396048 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fbdfc062630 con 0x7fbde40a3380 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (101s) 19s ago 100s 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c 6f89ad31d670 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi081 running (98s) 30s ago 97s 25.8M - 19.1.1-326-g8921c8ef f1455bd0219c 4fd20b705047 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi102 running (112s) 30s ago 112s 17.7M - 19.1.1-326-g8921c8ef f1455bd0219c 033e62d3d88d 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (2m) 19s ago 2m 15.9M - 19.1.1-326-g8921c8ef f1455bd0219c f61cf5472edb 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi081 running (118s) 30s ago 117s 16.9M - 19.1.1-326-g8921c8ef f1455bd0219c 3bb5c4de5f43 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi102 running (115s) 30s ago 114s 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c c996b89a561a 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (104s) 19s ago 103s 18.5M - 19.1.1-326-g8921c8ef f1455bd0219c 6b9e423ae81b 2024-09-15T23:59:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi081 running (110s) 30s ago 109s 19.9M - 19.1.1-326-g8921c8ef f1455bd0219c fa20050466ed 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi102 running (107s) 30s ago 106s 17.9M - 19.1.1-326-g8921c8ef f1455bd0219c 4531dd4e388b 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (2m) 19s ago 2m 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c cc669271f99b 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi081 running (2m) 30s ago 2m 18.5M - 19.1.1-326-g8921c8ef f1455bd0219c ad4fb911080b 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi102 running (2m) 30s ago 2m 16.9M - 19.1.1-326-g8921c8ef f1455bd0219c 0a7614f79f90 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 19s ago 9m 508M - 19.1.1-326-g8921c8ef f1455bd0219c b8edf6bca87f 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi081 *:8443,8765 running (7m) 30s ago 7m 441M - 19.1.1-326-g8921c8ef f1455bd0219c 1577a96482f2 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi102 *:8443,8765 running (7m) 30s ago 7m 443M - 19.1.1-326-g8921c8ef f1455bd0219c c74b0d77d51a 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 19s ago 9m 52.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 6c2bf311443b 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi081 running (7m) 30s ago 7m 44.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 94c27af25f7a 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi102 running (7m) 30s ago 7m 41.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ae05a1caf9ba 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 19s ago 7m 45.0M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 9410dbde6219 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi081 running (6m) 30s ago 6m 55.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c d6e5dc3ff2f3 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi102 running (6m) 30s ago 6m 48.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c ffb9ca2a640b 2024-09-15T23:59:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (5m) 19s ago 5m 56.7M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 56419a7ae595 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi081 running (5m) 30s ago 5m 43.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b427dc2ddbcc 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi102 running (5m) 30s ago 5m 45.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 123794bd807f 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 19s ago 4m 40.8M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 468a9312ffa5 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi081 running (4m) 30s ago 4m 54.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b8a6333e40d5 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi102 running (3m) 30s ago 3m 40.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0ff57c166170 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 19s ago 3m 43.7M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 8552a8aeb9a1 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi081 running (2m) 30s ago 2m 43.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0e674e27bc89 2024-09-15T23:59:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi102 running (2m) 30s ago 2m 40.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3916da660a8e 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 msgr2=0x7fbde40a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 0x7fbde40a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00058c0 tx=0x7fbdf0005850 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 msgr2=0x7fbdfc197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c970 tx=0x7fbdec00ce30 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 shutdown_connections 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fbde40a3380 0x7fbde40a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdfc107900 0x7fbdfc19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdfc104fd0 0x7fbdfc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 --2- 172.21.15.31:0/728396048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc1026a0 0x7fbdfc197e30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 >> 172.21.15.31:0/728396048 conn(0x7fbdfc0fc2a0 msgr2=0x7fbdfc103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:39.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 shutdown_connections 2024-09-15T23:59:39.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:39.214+0000 7fbe026ad640 1 -- 172.21.15.31:0/728396048 wait complete. 2024-09-15T23:59:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:39 smithi081 ceph-mon[35060]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:39 smithi081 ceph-mon[35060]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:39 smithi102 ceph-mon[34458]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-15T23:59:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:39 smithi102 ceph-mon[34458]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:39.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-15T23:59:40.202 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:40.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/2632161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f781037c0 msgr2=0x7f2f7810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/2632161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f781037c0 0x7f2f7810a050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009ca0 tx=0x7f2f7002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/2632161822 shutdown_connections 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/2632161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f781037c0 0x7f2f7810a050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/2632161822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f78102e00 0x7f2f78103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/2632161822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.693+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/2632161822 >> 172.21.15.31:0/2632161822 conn(0x7f2f780fd390 msgr2=0x7f2f780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/2632161822 shutdown_connections 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/2632161822 wait complete. 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 Processor -- start 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- start start 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:40.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 0x7f2f7819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7810d730 con 0x7f2f78102e00 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7810d5b0 con 0x7f2f78101c00 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f7ee4c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7810d8b0 con 0x7f2f781037c0 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.694+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52286/0 (socket says 172.21.15.31:52286) 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 -- 172.21.15.31:0/1201204458 learned_addr learned my addr 172.21.15.31:0/1201204458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7d3e9640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 0x7f2f7819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7cbe8640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 -- 172.21.15.31:0/1201204458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 msgr2=0x7f2f7819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 0x7f2f7819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 -- 172.21.15.31:0/1201204458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 msgr2=0x7f2f78197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 -- 172.21.15.31:0/1201204458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7819f9f0 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7d3e9640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 0x7f2f7819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7cbe8640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f77fff640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c970 tx=0x7f2f6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64007bb0 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f64007d50 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7819fce0 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400f450 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.695+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f781a01c0 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.696+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f78109050 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.697+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2f64004ee0 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.698+0000 7f2f75ffb640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 0x7f2f680a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.698+0000 7f2f7cbe8640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 0x7f2f680a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.698+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2f640cf690 con 0x7f2f78102e00 2024-09-15T23:59:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.699+0000 7f2f7cbe8640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 0x7f2f680a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f78102c60 tx=0x7f2f6c005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:40.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.702+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f64014030 con 0x7f2f78102e00 2024-09-15T23:59:40.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.887+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2f78062630 con 0x7f2f680a3380 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.894+0000 7f2f75ffb640 1 -- 172.21.15.31:0/1201204458 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2f78062630 con 0x7f2f680a3380 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 32s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 32s ago 7m smithi031=x;smithi081=y;smithi102=z;count:3 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 32s ago 8m smithi031:172.21.15.31=a;smithi081:172.21.15.81=b;smithi102:172.21.15.102=c;count:3 2024-09-15T23:59:40.896 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 32s ago - 2024-09-15T23:59:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 msgr2=0x7f2f680a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 0x7f2f680a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f78102c60 tx=0x7f2f6c005e70 comp rx=0 tx=0).stop 2024-09-15T23:59:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 msgr2=0x7f2f78198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c970 tx=0x7f2f6400ce30 comp rx=0 tx=0).stop 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 shutdown_connections 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f2f680a3380 0x7f2f680a5840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f781037c0 0x7f2f7819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f78102e00 0x7f2f78198300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 --2- 172.21.15.31:0/1201204458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f78101c00 0x7f2f78197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 >> 172.21.15.31:0/1201204458 conn(0x7f2f780fd390 msgr2=0x7f2f78106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 shutdown_connections 2024-09-15T23:59:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:40.897+0000 7f2f7ee4c640 1 -- 172.21.15.31:0/1201204458 wait complete. 2024-09-15T23:59:41.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:41 smithi031 ceph-mon[31203]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:41.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:41 smithi031 ceph-mon[31203]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:41.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:41 smithi081 ceph-mon[35060]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:41 smithi081 ceph-mon[35060]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:41.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:41 smithi102 ceph-mon[34458]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:41.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:41 smithi102 ceph-mon[34458]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-15T23:59:41.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-15T23:59:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:42.349 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:42 smithi031 ceph-mon[31203]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/770296523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb4102e00 msgr2=0x7f7cb4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/770296523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb4102e00 0x7f7cb4103280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009900 tx=0x7f7ca802a410 comp rx=0 tx=0).stop 2024-09-15T23:59:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/770296523 shutdown_connections 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/770296523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb41037c0 0x7f7cb410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/770296523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb4102e00 0x7f7cb4103280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/770296523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4101c00 0x7f7cb4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/770296523 >> 172.21.15.31:0/770296523 conn(0x7f7cb40fd390 msgr2=0x7f7cb40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/770296523 shutdown_connections 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.440+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/770296523 wait complete. 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 Processor -- start 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 -- start start 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb4101c00 0x7f7cb4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4102e00 0x7f7cb40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb410d710 con 0x7f7cb41037c0 2024-09-15T23:59:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb410d590 con 0x7f7cb4102e00 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb9bfa640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb410d890 con 0x7f7cb4101c00 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34616/0 (socket says 172.21.15.31:34616) 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb3fff640 1 -- 172.21.15.31:0/3632519063 learned_addr learned my addr 172.21.15.31:0/3632519063 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb2ffd640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4102e00 0x7f7cb40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.441+0000 7f7cb37fe640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb4101c00 0x7f7cb4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 -- 172.21.15.31:0/3632519063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb4101c00 msgr2=0x7f7cb4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb4101c00 0x7f7cb4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 -- 172.21.15.31:0/3632519063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4102e00 msgr2=0x7f7cb40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4102e00 0x7f7cb40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 -- 172.21.15.31:0/3632519063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb40753e0 con 0x7f7cb41037c0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb3fff640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cce0 tx=0x7f7cac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00f450 con 0x7f7cb41037c0 2024-09-15T23:59:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cac00faa0 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac018b70 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41a1f20 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.442+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a2380 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.443+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb4109020 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.444+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7cac018d10 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.445+0000 7f7cb0ff9640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 0x7f7c9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.445+0000 7f7cb37fe640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 0x7f7c9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.445+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7cac0cfa40 con 0x7f7cb41037c0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.445+0000 7f7cb37fe640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 0x7f7c9c0a5780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4005fd0 tx=0x7f7ca400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.449+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cac014030 con 0x7f7cb41037c0 2024-09-15T23:59:42.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:42 smithi081 ceph-mon[35060]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:42.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:42 smithi102 ceph-mon[34458]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.636+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7cb4062630 con 0x7f7c9c0a32c0 2024-09-15T23:59:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.638+0000 7f7cb0ff9640 1 -- 172.21.15.31:0/3632519063 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7cb4062630 con 0x7f7c9c0a32c0 2024-09-15T23:59:42.639 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-09-15T23:59:42.640 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-09-15T23:59:42.640 INFO:teuthology.orchestra.run.smithi031.stdout:smithi081 172.21.15.81 2024-09-15T23:59:42.640 INFO:teuthology.orchestra.run.smithi031.stdout:smithi102 172.21.15.102 2024-09-15T23:59:42.640 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 msgr2=0x7f7c9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 0x7f7c9c0a5780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4005fd0 tx=0x7f7ca400b410 comp rx=0 tx=0).stop 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 msgr2=0x7f7cb4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cce0 tx=0x7f7cac00a620 comp rx=0 tx=0).stop 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 shutdown_connections 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f7c9c0a32c0 0x7f7c9c0a5780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7cb41037c0 0x7f7cb4074b20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4102e00 0x7f7cb40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 --2- 172.21.15.31:0/3632519063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cb4101c00 0x7f7cb4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.640+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 >> 172.21.15.31:0/3632519063 conn(0x7f7cb40fd390 msgr2=0x7f7cb41061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.641+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 shutdown_connections 2024-09-15T23:59:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:42.641+0000 7f7cb9bfa640 1 -- 172.21.15.31:0/3632519063 wait complete. 2024-09-15T23:59:43.353 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:43 smithi031 ceph-mon[31203]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:43.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:43 smithi031 ceph-mon[31203]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-15T23:59:43.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-15T23:59:43.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:43 smithi081 ceph-mon[35060]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:43.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:43 smithi081 ceph-mon[35060]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-15T23:59:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:43 smithi102 ceph-mon[34458]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:43.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:43 smithi102 ceph-mon[34458]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-15T23:59:43.652 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:44.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 -- 172.21.15.31:0/2961433592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00100b10 msgr2=0x7fcc00100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/2961433592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00100b10 0x7fcc00100f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009900 tx=0x7fcbf002a410 comp rx=0 tx=0).stop 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 -- 172.21.15.31:0/2961433592 shutdown_connections 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/2961433592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc001026d0 0x7fcc00108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/2961433592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00101d10 0x7fcc00102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/2961433592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00100b10 0x7fcc00100f10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 -- 172.21.15.31:0/2961433592 >> 172.21.15.31:0/2961433592 conn(0x7fcc000fc2a0 msgr2=0x7fcc000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 -- 172.21.15.31:0/2961433592 shutdown_connections 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.132+0000 7fcc06ce5640 1 -- 172.21.15.31:0/2961433592 wait complete. 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 Processor -- start 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 -- start start 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00100b10 0x7fcc00196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc00101d10 0x7fcc00197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:44.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c620 con 0x7fcc001026d0 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c4a0 con 0x7fcc00101d10 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc06ce5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010c7a0 con 0x7fcc00100b10 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34620/0 (socket says 172.21.15.31:34620) 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 -- 172.21.15.31:0/280667594 learned_addr learned my addr 172.21.15.31:0/280667594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc04a81640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00100b10 0x7fcc00196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcbfffff640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc00101d10 0x7fcc00197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 -- 172.21.15.31:0/280667594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00100b10 msgr2=0x7fcc00196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00100b10 0x7fcc00196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 -- 172.21.15.31:0/280667594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc00101d10 msgr2=0x7fcc00197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc00101d10 0x7fcc00197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.133+0000 7fcc05282640 1 -- 172.21.15.31:0/280667594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0019e910 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.134+0000 7fcc05282640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8007900 tx=0x7fcbf8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.137+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800e6f0 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.137+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbf800ed40 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.137+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0006a7f0 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.137+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800ac40 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.137+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0006acb0 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.139+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc00100f10 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.140+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcbf800e890 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.140+0000 7fcbfdffb640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 0x7fcbe80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.141+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcbf80d1050 con 0x7fcc001026d0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.141+0000 7fcc04a81640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 0x7fcbe80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.141+0000 7fcc04a81640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 0x7fcbe80a5700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0004770 tx=0x7fcbf0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:44.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.147+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbf809dec0 con 0x7fcc001026d0 2024-09-15T23:59:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.332+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 --> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcc0006af10 con 0x7fcbe80a3240 2024-09-15T23:59:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:44 smithi031 ceph-mon[31203]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:44 smithi031 ceph-mon[31203]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.334+0000 7fcbfdffb640 1 -- 172.21.15.31:0/280667594 <== mgr.14150 v2:172.21.15.31:6800/797003742 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcc0006af10 con 0x7fcbe80a3240 2024-09-15T23:59:44.335 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-15T23:59:44.336 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T23:59:44.336 INFO:teuthology.orchestra.run.smithi031.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T23:59:44.336 INFO:teuthology.orchestra.run.smithi031.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T23:59:44.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.336+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 msgr2=0x7fcbe80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:44.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.336+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 0x7fcbe80a5700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0004770 tx=0x7fcbf0000bc0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.336+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 msgr2=0x7fcc0019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.336+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8007900 tx=0x7fcbf8007dc0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 shutdown_connections 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcbe80a3240 0x7fcbe80a5700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc001026d0 0x7fcc0019e270 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc00101d10 0x7fcc00197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 --2- 172.21.15.31:0/280667594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc00100b10 0x7fcc00196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 >> 172.21.15.31:0/280667594 conn(0x7fcc000fc2a0 msgr2=0x7fcc00105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 shutdown_connections 2024-09-15T23:59:44.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:44.337+0000 7fcc06ce5640 1 -- 172.21.15.31:0/280667594 wait complete. 2024-09-15T23:59:44.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:44 smithi081 ceph-mon[35060]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:44.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:44 smithi081 ceph-mon[35060]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:44.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:44 smithi102 ceph-mon[34458]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:44.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:44 smithi102 ceph-mon[34458]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:45.024 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T23:59:45.033 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-15T23:59:45.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-15T23:59:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:45 smithi031 ceph-mon[31203]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:45.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:45 smithi081 ceph-mon[35060]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:45.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:45 smithi102 ceph-mon[34458]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T23:59:45.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.801+0000 7f097cd5f640 1 -- 172.21.15.31:0/3152165710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 msgr2=0x7f0978109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.801+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3152165710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978109d50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0970009ca0 tx=0x7f097002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 -- 172.21.15.31:0/3152165710 shutdown_connections 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3152165710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978109d50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3152165710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0978104fd0 0x7f09781073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3152165710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09781026a0 0x7f0978104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 -- 172.21.15.31:0/3152165710 >> 172.21.15.31:0/3152165710 conn(0x7f09780fc2a0 msgr2=0x7f09780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 -- 172.21.15.31:0/3152165710 shutdown_connections 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 -- 172.21.15.31:0/3152165710 wait complete. 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 Processor -- start 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.802+0000 7f097cd5f640 1 -- start start 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09781026a0 0x7f0978102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0978104fd0 0x7f0978100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f097810d770 con 0x7f0978107900 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f097810d5f0 con 0x7f09781026a0 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097cd5f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f097810d8f0 con 0x7f0978104fd0 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34638/0 (socket says 172.21.15.31:34638) 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 -- 172.21.15.31:0/3842160663 learned_addr learned my addr 172.21.15.31:0/3842160663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f097659c640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09781026a0 0x7f0978102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0975d9b640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0978104fd0 0x7f0978100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 -- 172.21.15.31:0/3842160663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0978104fd0 msgr2=0x7f0978100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0978104fd0 0x7f0978100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 -- 172.21.15.31:0/3842160663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09781026a0 msgr2=0x7f0978102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09781026a0 0x7f0978102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 -- 172.21.15.31:0/3842160663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0978101450 con 0x7f0978107900 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f0976d9d640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0970009dc0 tx=0x7f0970003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.803+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970032ec0 con 0x7f0978107900 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.804+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09700044f0 con 0x7f0978107900 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.804+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a1fb0 con 0x7f0978107900 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.804+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970030d40 con 0x7f0978107900 2024-09-15T23:59:45.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.804+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a2460 con 0x7f0978107900 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.805+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09780fdd90 con 0x7f0978107900 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.806+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0970004040 con 0x7f0978107900 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.806+0000 7f09677fe640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 0x7f09600a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.807+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f09700f0e50 con 0x7f0978107900 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.807+0000 7f097659c640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 0x7f09600a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.807+0000 7f097659c640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 0x7f09600a58a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0968005fd0 tx=0x7f096800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:45.810+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09700bdcc0 con 0x7f0978107900 2024-09-15T23:59:46.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.024+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f097802d7c0 con 0x7f0978107900 2024-09-15T23:59:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.024+0000 7f09677fe640 1 -- 172.21.15.31:0/3842160663 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3120 (secure 0 0 0) 0x7f09700bd5c0 con 0x7f0978107900 2024-09-15T23:59:46.026 INFO:teuthology.orchestra.run.smithi031.stdout:e27 2024-09-15T23:59:46.026 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-15T23:58:24:929514+0000 2024-09-15T23:59:46.026 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T23:59:46.026 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 27 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-15T23:58:11.769531+0000 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-15T23:58:23.979593+0000 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-15T23:59:46.027 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 5 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2,3,4 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24491,1=24478,2=24461,3=24457,4=24451} 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-15T23:59:46.028 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24491 members: 24491,24478,24461,24457,24451 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24491} state up:active seq 4 addr [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{0:24434} state up:standby-replay seq 1 addr [v2:172.21.15.81:6834/799148008,v1:172.21.15.81:6835/799148008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{1:24478} state up:active seq 9 addr [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{1:24412} state up:standby-replay seq 1 addr [v2:172.21.15.102:6832/4135774449,v1:172.21.15.102:6833/4135774449] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{2:24461} state up:active seq 9 addr [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.81:6832/958737504,v1:172.21.15.81:6833/958737504] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{3:24457} state up:active seq 10 addr [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{3:14607} state up:standby-replay seq 1 addr [v2:172.21.15.31:6840/1341414618,v1:172.21.15.31:6841/1341414618] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{4:24451} state up:active seq 11 addr [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{4:14601} state up:standby-replay seq 1 addr [v2:172.21.15.31:6838/193010383,v1:172.21.15.31:6839/193010383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.31:6834/3861052716,v1:172.21.15.31:6835/3861052716] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.030 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.31:6836/735300624,v1:172.21.15.31:6837/735300624] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 msgr2=0x7f09600a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 0x7f09600a58a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0968005fd0 tx=0x7f096800b390 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 msgr2=0x7f0978100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0970009dc0 tx=0x7f0970003ba0 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 shutdown_connections 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f09600a33e0 0x7f09600a58a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0978107900 0x7f0978100b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0978104fd0 0x7f0978100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 --2- 172.21.15.31:0/3842160663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09781026a0 0x7f0978102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 >> 172.21.15.31:0/3842160663 conn(0x7f09780fc2a0 msgr2=0x7f0978103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 shutdown_connections 2024-09-15T23:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:46.026+0000 7f097cd5f640 1 -- 172.21.15.31:0/3842160663 wait complete. 2024-09-15T23:59:46.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:46 smithi031 ceph-mon[31203]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:46.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:46 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3842160663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T23:59:46.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:46 smithi081 ceph-mon[35060]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:46.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:46 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3842160663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T23:59:46.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:46 smithi102 ceph-mon[34458]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:46.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:46 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3842160663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T23:59:46.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-15T23:59:47.104 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/1368333378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8101cd0 msgr2=0x7fb8e8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/1368333378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8102150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009900 tx=0x7fb8d002a410 comp rx=0 tx=0).stop 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/1368333378 shutdown_connections 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/1368333378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8102690 0x7fb8e8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/1368333378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8102150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/1368333378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/1368333378 >> 172.21.15.31:0/1368333378 conn(0x7fb8e80fc280 msgr2=0x7fb8e80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/1368333378 shutdown_connections 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.606+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/1368333378 wait complete. 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 Processor -- start 2024-09-15T23:59:47.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 -- start start 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810c5e0 con 0x7fb8e8102690 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810c460 con 0x7fb8e8100ad0 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ee7c0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810c760 con 0x7fb8e8101cd0 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34654/0 (socket says 172.21.15.31:34654) 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 -- 172.21.15.31:0/3623002969 learned_addr learned my addr 172.21.15.31:0/3623002969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 -- 172.21.15.31:0/3623002969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 msgr2=0x7fb8e8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8dffff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 -- 172.21.15.31:0/3623002969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 msgr2=0x7fb8e8197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8ecd5d640 1 -- 172.21.15.31:0/3623002969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e819fa40 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.607+0000 7fb8dffff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8ecd5d640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009fd0 tx=0x7fb8e000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0017070 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e0003e20 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e819fd30 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0010460 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.608+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a01c0 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.609+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e81a03f0 con 0x7fb8e8102690 2024-09-15T23:59:47.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.610+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb8e0003970 con 0x7fb8e8102690 2024-09-15T23:59:47.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.611+0000 7fb8e5ffb640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 0x7fb8d40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:47.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.611+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 0x7fb8d40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:47.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.611+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb8e00cf660 con 0x7fb8e8102690 2024-09-15T23:59:47.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.611+0000 7fb8e7fff640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 0x7fb8d40a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8101b30 tx=0x7fb8d8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:47.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.615+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8e009c4d0 con 0x7fb8e8102690 2024-09-15T23:59:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.794+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb8e81a0750 con 0x7fb8e8102690 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.795+0000 7fb8e5ffb640 1 -- 172.21.15.31:0/3623002969 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5047 (secure 0 0 0) 0x7fb8e009bdd0 con 0x7fb8e8102690 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 80 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-15T23:50:23.938656+0000 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-15T23:58:11.769336+0000 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-09-15T23:59:47.797 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/1628537421,v1:172.21.15.31:6803/1628537421] [v2:172.21.15.31:6804/1628537421,v1:172.21.15.31:6805/1628537421] exists,up 1b544a11-7388-485f-9e4b-49e5bfa8459b 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/796455995,v1:172.21.15.81:6801/796455995] [v2:172.21.15.81:6802/796455995,v1:172.21.15.81:6803/796455995] exists,up cc0ccb76-ecd7-4c5b-b4f4-f282127b892a 2024-09-15T23:59:47.798 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/3162988112,v1:172.21.15.102:6801/3162988112] [v2:172.21.15.102:6802/3162988112,v1:172.21.15.102:6803/3162988112] exists,up 16d860f1-d8da-4d5d-ae50-10e77a4c4b2c 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/487246469,v1:172.21.15.31:6811/487246469] [v2:172.21.15.31:6812/487246469,v1:172.21.15.31:6813/487246469] exists,up 7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 31 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/3997776531,v1:172.21.15.81:6809/3997776531] [v2:172.21.15.81:6810/3997776531,v1:172.21.15.81:6811/3997776531] exists,up f1085071-44fa-4277-b09b-7e5ef3b45330 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 36 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/3259467463,v1:172.21.15.102:6809/3259467463] [v2:172.21.15.102:6810/3259467463,v1:172.21.15.102:6811/3259467463] exists,up 010742fa-38fa-4483-9f2f-173c0fd11a06 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 43 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/515354645,v1:172.21.15.31:6819/515354645] [v2:172.21.15.31:6820/515354645,v1:172.21.15.31:6821/515354645] exists,up d7c78c23-60ea-4bb6-b269-9d236295e6b4 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 48 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/326400233,v1:172.21.15.81:6817/326400233] [v2:172.21.15.81:6818/326400233,v1:172.21.15.81:6819/326400233] exists,up f270c537-d25a-4ee7-87a0-b18f67a00a2c 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 55 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/3203076122,v1:172.21.15.102:6817/3203076122] [v2:172.21.15.102:6818/3203076122,v1:172.21.15.102:6819/3203076122] exists,up 530bcf2d-1d5c-4ad2-bba2-fad10d3e581e 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 62 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1878664513,v1:172.21.15.31:6827/1878664513] [v2:172.21.15.31:6828/1878664513,v1:172.21.15.31:6829/1878664513] exists,up d1bf1827-c5c0-4b3b-9f97-f7eb1679d742 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6824/397827149,v1:172.21.15.81:6825/397827149] [v2:172.21.15.81:6826/397827149,v1:172.21.15.81:6827/397827149] exists,up a086c66d-46cc-4b3a-a8cf-7022a6acf89e 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 74 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/4272602493,v1:172.21.15.102:6825/4272602493] [v2:172.21.15.102:6826/4272602493,v1:172.21.15.102:6827/4272602493] exists,up 3eead23d-bbb5-4fd5-adc7-3330798d55f4 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/108921567 expires 2024-09-16T23:51:11.105458+0000 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2022751094 expires 2024-09-16T23:51:11.105458+0000 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/639707976 expires 2024-09-16T23:51:11.105458+0000 2024-09-15T23:59:47.799 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3011001440 expires 2024-09-16T23:51:11.105458+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/639707976 expires 2024-09-16T23:51:11.105458+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3308606376 expires 2024-09-16T23:50:46.251673+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/741219641 expires 2024-09-16T23:50:46.251673+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1530295855 expires 2024-09-16T23:50:46.251673+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1530295855 expires 2024-09-16T23:50:46.251673+0000 2024-09-15T23:59:47.800 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2347317772 expires 2024-09-16T23:50:46.251673+0000 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 msgr2=0x7fb8d40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 0x7fb8d40a5730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8101b30 tx=0x7fb8d8005d30 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 msgr2=0x7fb8e819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009fd0 tx=0x7fb8e000b260 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 shutdown_connections 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.797+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fb8d40a3270 0x7fb8d40a5730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e8102690 0x7fb8e819f3d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8e8101cd0 0x7fb8e8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 --2- 172.21.15.31:0/3623002969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8e8100ad0 0x7fb8e8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 >> 172.21.15.31:0/3623002969 conn(0x7fb8e80fc280 msgr2=0x7fb8e806a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 shutdown_connections 2024-09-15T23:59:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:47.798+0000 7fb8ee7c0640 1 -- 172.21.15.31:0/3623002969 wait complete. 2024-09-15T23:59:48.457 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-15T23:59:48.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:59:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:48 smithi081 ceph-mon[35060]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:48.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:48 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3623002969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T23:59:48.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:48 smithi102 ceph-mon[34458]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:48.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:48 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3623002969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T23:59:48.726 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:48.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:48 smithi031 ceph-mon[31203]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:48.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:48 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3623002969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T23:59:49.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 -- 172.21.15.31:0/3708522533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38101de0 msgr2=0x7f1b381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 --2- 172.21.15.31:0/3708522533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38101de0 0x7f1b381021c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009900 tx=0x7f1b3402a410 comp rx=0 tx=0).stop 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 -- 172.21.15.31:0/3708522533 shutdown_connections 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 --2- 172.21.15.31:0/3708522533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38106f10 0x7f1b3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 --2- 172.21.15.31:0/3708522533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 --2- 172.21.15.31:0/3708522533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38101de0 0x7f1b381021c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 -- 172.21.15.31:0/3708522533 >> 172.21.15.31:0/3708522533 conn(0x7f1b380fd8c0 msgr2=0x7f1b380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 -- 172.21.15.31:0/3708522533 shutdown_connections 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.212+0000 7f1b3f951640 1 -- 172.21.15.31:0/3708522533 wait complete. 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 Processor -- start 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 -- start start 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 0x7f1b380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:49.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38111e20 con 0x7f1b38106f10 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38111ca0 con 0x7f1b38101de0 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3f951640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38111fa0 con 0x7f1b38102790 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34668/0 (socket says 172.21.15.31:34668) 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 -- 172.21.15.31:0/2321270724 learned_addr learned my addr 172.21.15.31:0/2321270724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3d6ed640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 0x7f1b380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3ceec640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 -- 172.21.15.31:0/2321270724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 msgr2=0x7f1b38074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b38074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 -- 172.21.15.31:0/2321270724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 msgr2=0x7f1b380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 0x7f1b380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 -- 172.21.15.31:0/2321270724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b381a6840 con 0x7f1b38106f10 2024-09-15T23:59:49.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3ceec640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3d6ed640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 0x7f1b380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.213+0000 7f1b3deee640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cad0 tx=0x7f1b3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.214+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000ae20 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.214+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b381a6b30 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.214+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b30005020 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.214+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000f6b0 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.214+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a7020 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.215+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b381021c0 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.216+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1b300051d0 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.216+0000 7f1b2e7fc640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 0x7f1b240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.217+0000 7f1b3d6ed640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 0x7f1b240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.217+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1b300cf900 con 0x7f1b38106f10 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.217+0000 7f1b3d6ed640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 0x7f1b240a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b34031440 tx=0x7f1b340095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:49.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.221+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b30014030 con 0x7f1b38106f10 2024-09-15T23:59:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.434+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b3810a0f0 con 0x7f1b38106f10 2024-09-15T23:59:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.435+0000 7f1b2e7fc640 1 -- 172.21.15.31:0/2321270724 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7f1b3009c6d0 con 0x7f1b38106f10 2024-09-15T23:59:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:49.437 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:59:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 msgr2=0x7f1b240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 0x7f1b240a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1b34031440 tx=0x7f1b340095f0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 msgr2=0x7f1b38074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cad0 tx=0x7f1b3000cf90 comp rx=0 tx=0).stop 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 shutdown_connections 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1b240a3240 0x7f1b240a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106f10 0x7f1b38074f70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b38102790 0x7f1b38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 --2- 172.21.15.31:0/2321270724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b38101de0 0x7f1b380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.437+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 >> 172.21.15.31:0/2321270724 conn(0x7f1b380fd8c0 msgr2=0x7f1b380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.438+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 shutdown_connections 2024-09-15T23:59:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:49.438+0000 7f1b3f951640 1 -- 172.21.15.31:0/2321270724 wait complete. 2024-09-15T23:59:50.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:59:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:50.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:50 smithi031 ceph-mon[31203]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-15T23:59:50.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:50 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2321270724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:50.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:50 smithi081 ceph-mon[35060]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-15T23:59:50.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:50 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2321270724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:50.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:50 smithi102 ceph-mon[34458]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-15T23:59:50.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:50 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2321270724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.822+0000 7f902611e640 1 -- 172.21.15.31:0/1069781974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 msgr2=0x7f9020106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.822+0000 7f902611e640 1 --2- 172.21.15.31:0/1069781974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f9020106e60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9010009930 tx=0x7f901002a420 comp rx=0 tx=0).stop 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 -- 172.21.15.31:0/1069781974 shutdown_connections 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 --2- 172.21.15.31:0/1069781974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020101420 0x7f902010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 --2- 172.21.15.31:0/1069781974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020100a80 0x7f9020100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 --2- 172.21.15.31:0/1069781974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f9020106e60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 -- 172.21.15.31:0/1069781974 >> 172.21.15.31:0/1069781974 conn(0x7f90200fc7a0 msgr2=0x7f90200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 -- 172.21.15.31:0/1069781974 shutdown_connections 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 -- 172.21.15.31:0/1069781974 wait complete. 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 Processor -- start 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 -- start start 2024-09-15T23:59:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100a80 0x7f90201983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.823+0000 7f902611e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 0x7f90201988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9020110d30 con 0x7f9020106a80 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9020110bb0 con 0x7f9020101420 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9020110eb0 con 0x7f9020100a80 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34686/0 (socket says 172.21.15.31:34686) 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 -- 172.21.15.31:0/2930632388 learned_addr learned my addr 172.21.15.31:0/2930632388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901f7fe640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100a80 0x7f90201983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f9017fff640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 0x7f90201988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:50.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 -- 172.21.15.31:0/2930632388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100a80 msgr2=0x7f90201983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100a80 0x7f90201983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 -- 172.21.15.31:0/2930632388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 msgr2=0x7f90201988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 0x7f90201988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 -- 172.21.15.31:0/2930632388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902019d360 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f9017fff640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 0x7f90201988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901ffff640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f901800eaa0 tx=0x7f901800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901800cdb0 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9018005020 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902019d650 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90180116d0 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.824+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90201a4f30 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.825+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9020100ee0 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.826+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f90180051d0 con 0x7f9020106a80 2024-09-15T23:59:50.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.827+0000 7f901d7fa640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.827+0000 7f901f7fe640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.827+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f90180cf920 con 0x7f9020106a80 2024-09-15T23:59:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.828+0000 7f901f7fe640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 0x7f900c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9010009900 tx=0x7f9010000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:50.831+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9018016030 con 0x7f9020106a80 2024-09-15T23:59:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.041+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9020109a70 con 0x7f9020106a80 2024-09-15T23:59:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.042+0000 7f901d7fa640 1 -- 172.21.15.31:0/2930632388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11961 (secure 0 0 0) 0x7f901809c790 con 0x7f9020106a80 2024-09-15T23:59:51.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:51.044 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-15T23:58:24:929514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-15T23:58:23.979593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24478,24461,24457,24451]},"id":1}]} 2024-09-15T23:59:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 msgr2=0x7f900c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 0x7f900c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9010009900 tx=0x7f9010000bc0 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 msgr2=0x7f902019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f901800eaa0 tx=0x7f901800ef60 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 shutdown_connections 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f900c0a3240 0x7f900c0a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9020106a80 0x7f902019cc80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9020101420 0x7f90201988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 --2- 172.21.15.31:0/2930632388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9020100a80 0x7f90201983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 >> 172.21.15.31:0/2930632388 conn(0x7f90200fc7a0 msgr2=0x7f902006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 shutdown_connections 2024-09-15T23:59:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:51.044+0000 7f902611e640 1 -- 172.21.15.31:0/2930632388 wait complete. 2024-09-15T23:59:51.353 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:51 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2930632388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:51.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:51 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2930632388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:51.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:51 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2930632388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:59:51.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:59:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:52.358 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:52 smithi031 ceph-mon[31203]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 -- 172.21.15.31:0/2814235539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 msgr2=0x7fc3cc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 --2- 172.21.15.31:0/2814235539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1031b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009930 tx=0x7fc3c802a420 comp rx=0 tx=0).stop 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 -- 172.21.15.31:0/2814235539 shutdown_connections 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 --2- 172.21.15.31:0/2814235539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc108300 0x7fc3cc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 --2- 172.21.15.31:0/2814235539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc103780 0x7fc3cc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 --2- 172.21.15.31:0/2814235539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1031b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 -- 172.21.15.31:0/2814235539 >> 172.21.15.31:0/2814235539 conn(0x7fc3cc0fc7d0 msgr2=0x7fc3cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 -- 172.21.15.31:0/2814235539 shutdown_connections 2024-09-15T23:59:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.437+0000 7fc3d476a640 1 -- 172.21.15.31:0/2814235539 wait complete. 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 Processor -- start 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 -- start start 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc103780 0x7fc3cc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc108300 0x7fc3cc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc111f60 con 0x7fc3cc102dd0 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc111de0 con 0x7fc3cc108300 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d476a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1120e0 con 0x7fc3cc103780 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40362/0 (socket says 172.21.15.31:40362) 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 -- 172.21.15.31:0/3532988709 learned_addr learned my addr 172.21.15.31:0/3532988709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2d07640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc108300 0x7fc3cc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 -- 172.21.15.31:0/3532988709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc103780 msgr2=0x7fc3cc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d1d05640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc103780 0x7fc3cc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc103780 0x7fc3cc198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 -- 172.21.15.31:0/3532988709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc108300 msgr2=0x7fc3cc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc108300 0x7fc3cc19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.438+0000 7fc3d2506640 1 -- 172.21.15.31:0/3532988709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc19d490 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3d2506640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004770 tx=0x7fc3c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c803f070 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc19d720 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3c803b460 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8035070 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.439+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1a5060 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.440+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc1031b0 con 0x7fc3cc102dd0 2024-09-15T23:59:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.441+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc3c803b610 con 0x7fc3cc102dd0 2024-09-15T23:59:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.442+0000 7fc3c37fe640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 0x7fc3b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.442+0000 7fc3d1d05640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 0x7fc3b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.442+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc3c80efe60 con 0x7fc3cc102dd0 2024-09-15T23:59:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.442+0000 7fc3d1d05640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 0x7fc3b80a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005fd0 tx=0x7fc3bc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.445+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3c80bccd0 con 0x7fc3cc102dd0 2024-09-15T23:59:52.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:52 smithi081 ceph-mon[35060]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:52.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:52 smithi102 ceph-mon[34458]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-15T23:59:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.620+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3cc10a290 con 0x7fc3cc102dd0 2024-09-15T23:59:52.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.622+0000 7fc3c37fe640 1 -- 172.21.15.31:0/3532988709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22217 (secure 0 0 0) 0x7fc3c80bc5d0 con 0x7fc3cc102dd0 2024-09-15T23:59:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-15T23:59:52.624 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-15T23:58:11.769336+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 msgr2=0x7fc3b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 0x7fc3b80a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005fd0 tx=0x7fc3bc005850 comp rx=0 tx=0).stop 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 msgr2=0x7fc3cc1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004770 tx=0x7fc3c80042f0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 shutdown_connections 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fc3b80a3240 0x7fc3b80a5700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3cc108300 0x7fc3cc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3cc103780 0x7fc3cc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 --2- 172.21.15.31:0/3532988709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc102dd0 0x7fc3cc1984e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:52.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 >> 172.21.15.31:0/3532988709 conn(0x7fc3cc0fc7d0 msgr2=0x7fc3cc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:52.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 shutdown_connections 2024-09-15T23:59:52.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:52.624+0000 7fc3d476a640 1 -- 172.21.15.31:0/3532988709 wait complete. 2024-09-15T23:59:53.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:53 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3532988709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:53.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T23:58:11.769531+0000', 'modified': '2024-09-15T23:58:23.979593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24491, 'mds_1': 24478, 'mds_2': 24461, 'mds_3': 24457, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/193010383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 193010383}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 193010383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14607': {'gid': 14607, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/1341414618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 1341414618}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 1341414618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/958737504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 958737504}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 958737504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6833/4135774449', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 4135774449}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 4135774449}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6835/799148008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 799148008}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 799148008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.102:6835/2759418980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2759418980}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2759418980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.102:6837/4082772592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 4082772592}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 4082772592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.81:6837/3184710981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3184710981}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3184710981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.102:6839/3686122257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3686122257}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3686122257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6839/471643993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 471643993}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 471643993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24491, 24478, 24461, 24457, 24451]} 2024-09-15T23:59:53.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T23:59:53.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-15T23:59:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:53 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3532988709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:53 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3532988709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:59:53.638 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 -- 172.21.15.31:0/3083481918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730100e20 msgr2=0x7f6730101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 --2- 172.21.15.31:0/3083481918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730100e20 0x7f6730101200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6720009900 tx=0x7f672002a480 comp rx=0 tx=0).stop 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 -- 172.21.15.31:0/3083481918 shutdown_connections 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 --2- 172.21.15.31:0/3083481918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730102ef0 0x7f673010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 --2- 172.21.15.31:0/3083481918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102570 0x7f67301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 --2- 172.21.15.31:0/3083481918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730100e20 0x7f6730101200 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 -- 172.21.15.31:0/3083481918 >> 172.21.15.31:0/3083481918 conn(0x7f67300fc7d0 msgr2=0x7f67300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 -- 172.21.15.31:0/3083481918 shutdown_connections 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.132+0000 7f6735285640 1 -- 172.21.15.31:0/3083481918 wait complete. 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 Processor -- start 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 -- start start 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730100e20 0x7f67301992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 0x7f67301a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6730110ea0 con 0x7f6730102570 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6730110d20 con 0x7f6730102ef0 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f6735285640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6730111020 con 0x7f6730100e20 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f672e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f672e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40380/0 (socket says 172.21.15.31:40380) 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.133+0000 7f672e7fc640 1 -- 172.21.15.31:0/4007490239 learned_addr learned my addr 172.21.15.31:0/4007490239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:54.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672effd640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730100e20 0x7f67301992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672f7fe640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 0x7f67301a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 -- 172.21.15.31:0/4007490239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730100e20 msgr2=0x7f67301992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730100e20 0x7f67301992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 -- 172.21.15.31:0/4007490239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 msgr2=0x7f67301a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 0x7f67301a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 -- 172.21.15.31:0/4007490239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67301a4890 con 0x7f6730102570 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672f7fe640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 0x7f67301a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f672e7fc640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9f0 tx=0x7f672400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f6717fff640 1 -- 172.21.15.31:0/4007490239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672400cd80 con 0x7f6730102570 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a4a60 con 0x7f6730102570 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f6717fff640 1 -- 172.21.15.31:0/4007490239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67240108e0 con 0x7f6730102570 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f6717fff640 1 -- 172.21.15.31:0/4007490239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724018920 con 0x7f6730102570 2024-09-15T23:59:54.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.134+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a4fa0 con 0x7f6730102570 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.136+0000 7f6717fff640 1 -- 172.21.15.31:0/4007490239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6724010a90 con 0x7f6730102570 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f6717fff640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 0x7f67180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 Processor -- start 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f672effd640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 0x7f67180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f6717fff640 1 -- 172.21.15.31:0/4007490239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f67240cfc90 con 0x7f6730102570 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 -- start start 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f81f0 0x5581b1306a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b13093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581b125e090 con 0x5581b12f78d0 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5581b1205a70 con 0x5581b12f81f0 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.137+0000 7f673765e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581b125a050 con 0x5581b1306fc0 2024-09-15T23:59:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40394/0 (socket says 172.21.15.31:40394) 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 -- 172.21.15.31:0/344779826 learned_addr learned my addr 172.21.15.31:0/344779826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672e7fc640 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f81f0 0x5581b1306a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 0x7f67180a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f672002a9a0 tx=0x7f6720005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672f7fe640 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b13093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 -- 172.21.15.31:0/344779826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 msgr2=0x5581b13093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b13093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 -- 172.21.15.31:0/344779826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f81f0 msgr2=0x5581b1306a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f81f0 0x5581b1306a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 -- 172.21.15.31:0/344779826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b1212130 con 0x5581b12f78d0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f672effd640 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6720004610 tx=0x7f672003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2a41bf03df9301 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f67157fa640 1 -- 172.21.15.31:0/344779826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720035050 con 0x5581b12f78d0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f67157fa640 1 -- 172.21.15.31:0/344779826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f672004d070 con 0x5581b12f78d0 2024-09-15T23:59:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.138+0000 7f67157fa640 1 -- 172.21.15.31:0/344779826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003e10 con 0x5581b12f78d0 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- 172.21.15.31:0/344779826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 msgr2=0x5581b12f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6720004610 tx=0x7f672003b000 comp rx=0 tx=0).stop 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- 172.21.15.31:0/344779826 shutdown_connections 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b13093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f81f0 0x5581b1306a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 --2- 172.21.15.31:0/344779826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f7cb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- 172.21.15.31:0/344779826 >> 172.21.15.31:0/344779826 conn(0x5581b12f3f60 msgr2=0x5581b12f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- 172.21.15.31:0/344779826 shutdown_connections 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- 172.21.15.31:0/344779826 wait complete. 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 -- start start 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.139+0000 7f673765e740 1 Processor -- start 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b1306870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1303bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581b1309b90 con 0x5581b1306fc0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5581b1218f20 con 0x5581b12f78d0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581b1309d10 con 0x5581b12f81f0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 10 client.0 _ll_register_callbacks cb 0x5581b11e2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b1306870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40404/0 (socket says 172.21.15.31:40404) 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 -- 172.21.15.31:0/2894666051 learned_addr learned my addr 172.21.15.31:0/2894666051 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672e7fc640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1303bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 -- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 msgr2=0x5581b1303bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1303bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 -- 172.21.15.31:0/2894666051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 msgr2=0x5581b1306870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b1306870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 -- 172.21.15.31:0/2894666051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b0fcf2b0 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672e7fc640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1303bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f672f7fe640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f672800eb00 tx=0x7f6728007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c743a84b39d8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f6712ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800ce60 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6728010420 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.140+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672004d070 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6728023050 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 10 client.15114 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581b13155a0 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f672004d070 con 0x5581b1306fc0 2024-09-15T23:59:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f672800f6e0 con 0x5581b1306fc0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 20 client.15114 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b1317bc0 0x5581b1319030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 4 client.15114 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 --> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b126b600 con 0x5581b1317bc0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.141+0000 7f672effd640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b1317bc0 0x5581b1319030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.142+0000 7f672effd640 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b1317bc0 0x5581b1319030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7cd675e0d451fe56 server_cookie=16f3895241ab428b in_seq=0 out_seq=0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.142+0000 7f6712ff5640 10 client.15114 ms_handle_connect on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.142+0000 7f6712ff5640 1 -- 172.21.15.31:0/2894666051 <== mds.4 v2:172.21.15.102:6834/2759418980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581b126b600 con 0x5581b1317bc0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.142+0000 7f6712ff5640 10 client.15114 handle_command_reply: tid=0 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.142+0000 7f673765e740 1 client.15114 shutdown 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 msgr2=0x5581b1304100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f672800eb00 tx=0x7f6728007f60 comp rx=0 tx=0).stop 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 shutdown_connections 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b1317bc0 0x5581b1319030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b1306fc0 0x5581b1304100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1303bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 --2- 172.21.15.31:0/2894666051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b1306870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f6712ff5640 0 client.15114 ms_handle_reset on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 >> 172.21.15.31:0/2894666051 conn(0x5581b12f3f60 msgr2=0x5581b12f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 shutdown_connections 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 1 -- 172.21.15.31:0/2894666051 wait complete. 2024-09-15T23:59:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.143+0000 7f673765e740 20 client.15114 trim_cache size 0 max 16384 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 Processor -- start 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 -- start start 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1300e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b1306fc0 0x5581b1301360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581b1317370 con 0x5581b12f78d0 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5581b1317a40 con 0x5581b1306fc0 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f673765e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581b1318110 con 0x5581b12f81f0 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40416/0 (socket says 172.21.15.31:40416) 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 -- 172.21.15.31:0/1292074008 learned_addr learned my addr 172.21.15.31:0/1292074008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672e7fc640 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1300e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672f7fe640 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b1306fc0 0x5581b1301360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 -- 172.21.15.31:0/1292074008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 msgr2=0x5581b1300e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1300e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 -- 172.21.15.31:0/1292074008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b1306fc0 msgr2=0x5581b1301360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b1306fc0 0x5581b1301360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.161+0000 7f672effd640 1 -- 172.21.15.31:0/1292074008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b1309d00 con 0x5581b12f78d0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f672effd640 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6720004170 tx=0x7f672003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2eefd5744f0795 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f6714ff9640 1 -- 172.21.15.31:0/1292074008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720060070 con 0x5581b12f78d0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f6714ff9640 1 -- 172.21.15.31:0/1292074008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f672004d070 con 0x5581b12f78d0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f6714ff9640 1 -- 172.21.15.31:0/1292074008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720035040 con 0x5581b12f78d0 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 -- 172.21.15.31:0/1292074008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 msgr2=0x5581b12f8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6720004170 tx=0x7f672003d000 comp rx=0 tx=0).stop 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 -- 172.21.15.31:0/1292074008 shutdown_connections 2024-09-15T23:59:54.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b1306fc0 0x5581b1301360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b12f81f0 0x5581b1300e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 --2- 172.21.15.31:0/1292074008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f78d0 0x5581b12f8a20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 -- 172.21.15.31:0/1292074008 >> 172.21.15.31:0/1292074008 conn(0x5581b12f3f60 msgr2=0x5581b1302490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 -- 172.21.15.31:0/1292074008 shutdown_connections 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.162+0000 7f673765e740 1 -- 172.21.15.31:0/1292074008 wait complete. 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 -- start start 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 Processor -- start 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b131a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b131abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581b1318eb0 con 0x5581b12f81f0 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5581b13187e0 con 0x5581b12f78d0 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581b1319580 con 0x5581b1306fc0 2024-09-15T23:59:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f673765e740 10 client.0 _ll_register_callbacks cb 0x7f6728007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f672e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f672e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40420/0 (socket says 172.21.15.31:40420) 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f672e7fc640 1 -- 172.21.15.31:0/567555056 learned_addr learned my addr 172.21.15.31:0/567555056 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f672f7fe640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b131abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.163+0000 7f672effd640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b131a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 -- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 msgr2=0x5581b131abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b131abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 -- 172.21.15.31:0/567555056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 msgr2=0x5581b131a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b131a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 -- 172.21.15.31:0/567555056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b13802c0 con 0x5581b12f81f0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672effd640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b131a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672f7fe640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b131abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f672e7fc640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f672409c2a0 tx=0x7f6724098600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18420496cb6ef32f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240b4020 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6724010430 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672004d070 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67240a4fd0 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 10 client.15120 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581b1381590 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f672004d070 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f67240a0370 con 0x5581b12f81f0 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 10 client.15120 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 20 client.15120 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.164+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b13860e0 0x5581b1384df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:54.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 4 client.15120 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 --> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] -- command(tid 0: {"prefix": "status"}) -- 0x5581b1265b10 con 0x5581b13860e0 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f672effd640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b13860e0 0x5581b1384df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f6712ff5640 10 client.15120 ms_handle_connect on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f672effd640 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b13860e0 0x5581b1384df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=924235e028b79451 server_cookie=305fc1847bb572b8 in_seq=0 out_seq=0 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f6712ff5640 1 -- 172.21.15.31:0/567555056 <== mds.4 v2:172.21.15.102:6834/2759418980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5581b1265b10 con 0x5581b13860e0 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f6712ff5640 10 client.15120 handle_command_reply: tid=0 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 client.15120 shutdown 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 msgr2=0x5581b131a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f672409c2a0 tx=0x7f6724098600 comp rx=0 tx=0).stop 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 shutdown_connections 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:6834/2759418980,v1:172.21.15.102:6835/2759418980] conn(0x5581b13860e0 0x5581b1384df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581b1306fc0 0x5581b131abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581b12f81f0 0x5581b131a680 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 --2- 172.21.15.31:0/567555056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581b12f78d0 0x5581b131a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 >> 172.21.15.31:0/567555056 conn(0x5581b12f3f60 msgr2=0x5581b1302400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.165+0000 7f6712ff5640 0 client.15120 ms_handle_reset on v2:172.21.15.102:6834/2759418980 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 shutdown_connections 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f673765e740 1 -- 172.21.15.31:0/567555056 wait complete. 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f673765e740 20 client.15120 trim_cache size 0 max 16384 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24451, 2024-09-15T23:59:54.178 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 90.876119118000005, 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 129.910638041 2024-09-15T23:59:54.179 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 msgr2=0x7f67180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 0x7f67180a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f672002a9a0 tx=0x7f6720005fd0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 msgr2=0x7f6730199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f672400e9f0 tx=0x7f672400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 shutdown_connections 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f67180a3240 0x7f67180a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6730102ef0 0x7f67301a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730102570 0x7f6730199810 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 --2- 172.21.15.31:0/4007490239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6730100e20 0x7f67301992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 >> 172.21.15.31:0/4007490239 conn(0x7f67300fc7d0 msgr2=0x7f67300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.166+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 shutdown_connections 2024-09-15T23:59:54.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:54.167+0000 7f6735285640 1 -- 172.21.15.31:0/4007490239 wait complete. 2024-09-15T23:59:54.487 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:54 smithi031 ceph-mon[31203]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:54.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:54 smithi081 ceph-mon[35060]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:54.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:54 smithi102 ceph-mon[34458]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-15T23:59:55.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-15T23:59:55.279 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:55.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 -- 172.21.15.31:0/3963531813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 msgr2=0x7f1f401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 --2- 172.21.15.31:0/3963531813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401067e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009900 tx=0x7f1f3402a410 comp rx=0 tx=0).stop 2024-09-15T23:59:55.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 -- 172.21.15.31:0/3963531813 shutdown_connections 2024-09-15T23:59:55.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 --2- 172.21.15.31:0/3963531813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 --2- 172.21.15.31:0/3963531813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401067e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 --2- 172.21.15.31:0/3963531813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 0x7f1f401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 -- 172.21.15.31:0/3963531813 >> 172.21.15.31:0/3963531813 conn(0x7f1f400fd8c0 msgr2=0x7f1f400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 -- 172.21.15.31:0/3963531813 shutdown_connections 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 -- 172.21.15.31:0/3963531813 wait complete. 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.742+0000 7f1f4676a640 1 Processor -- start 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 -- start start 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 0x7f1f401983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111e20 con 0x7f1f40102790 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111ca0 con 0x7f1f40106f10 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f4676a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111fa0 con 0x7f1f40101de0 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40450/0 (socket says 172.21.15.31:40450) 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1737217179 learned_addr learned my addr 172.21.15.31:0/1737217179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 0x7f1f401983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1737217179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 msgr2=0x7f1f401983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 0x7f1f401983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1737217179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 msgr2=0x7f1f4019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1737217179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4019d350 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.743+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005940 tx=0x7f1f34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f3d7fa640 1 -- 172.21.15.31:0/1737217179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3403f030 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4019d5e0 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f3d7fa640 1 -- 172.21.15.31:0/1737217179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f34003aa0 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f3d7fa640 1 -- 172.21.15.31:0/1737217179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34035070 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.744+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401a4f20 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.746+0000 7f1f3d7fa640 1 -- 172.21.15.31:0/1737217179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1f34042030 con 0x7f1f40102790 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 Processor -- start 2024-09-15T23:59:55.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 -- start start 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 0x55f95c05d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 0x55f95c05ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f95bfb9090 con 0x55f95c0529d0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f95bf60a70 con 0x55f95c0532f0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f48b43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f95bfb5050 con 0x55f95c05dbb0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f3d7fa640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 0x7f1f280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40464/0 (socket says 172.21.15.31:40464) 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 -- 172.21.15.31:0/2079685520 learned_addr learned my addr 172.21.15.31:0/2079685520 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 0x7f1f280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f3d7fa640 1 -- 172.21.15.31:0/1737217179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1f340effb0 con 0x7f1f40102790 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 0x55f95c05d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 0x55f95c05ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 -- 172.21.15.31:0/2079685520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 msgr2=0x55f95c05ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.747+0000 7f1f44d07640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 0x55f95c05ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f44d07640 1 -- 172.21.15.31:0/2079685520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 msgr2=0x55f95c05d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f44d07640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 0x55f95c05d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f44d07640 1 -- 172.21.15.31:0/2079685520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95bf6d130 con 0x55f95c0529d0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 0x55f95c05d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 0x55f95c05ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f44d07640 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f38012920 tx=0x7f1f38012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6c926a7052357d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f267fc640 1 -- 172.21.15.31:0/2079685520 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38022070 con 0x55f95c0529d0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f267fc640 1 -- 172.21.15.31:0/2079685520 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f38012030 con 0x55f95c0529d0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.748+0000 7f1f267fc640 1 -- 172.21.15.31:0/2079685520 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38005280 con 0x55f95c0529d0 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- 172.21.15.31:0/2079685520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 msgr2=0x55f95c052db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 0x7f1f280a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f38020000 tx=0x7f1f3801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f38012920 tx=0x7f1f38012de0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- 172.21.15.31:0/2079685520 shutdown_connections 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c05dbb0 0x55f95c05ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0532f0 0x55f95c05d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 --2- 172.21.15.31:0/2079685520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c052db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- 172.21.15.31:0/2079685520 >> 172.21.15.31:0/2079685520 conn(0x55f95c04ef60 msgr2=0x55f95c04f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- 172.21.15.31:0/2079685520 shutdown_connections 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- 172.21.15.31:0/2079685520 wait complete. 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 -- start start 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.749+0000 7f1f48b43740 1 Processor -- start 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0529d0 0x55f95c05d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c05a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f95c060780 con 0x55f95c05dbb0 2024-09-15T23:59:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f95bf73f20 con 0x55f95c0529d0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f95c060900 con 0x55f95c0532f0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 10 client.0 _ll_register_callbacks cb 0x55f95bf3d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40466/0 (socket says 172.21.15.31:40466) 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 -- 172.21.15.31:0/4064616549 learned_addr learned my addr 172.21.15.31:0/4064616549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c05a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0529d0 0x55f95c05d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 -- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 msgr2=0x55f95c05a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c05a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 -- 172.21.15.31:0/4064616549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0529d0 msgr2=0x55f95c05d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0529d0 0x55f95c05d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 -- 172.21.15.31:0/4064616549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95bd2a2b0 con 0x55f95c05dbb0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f44d07640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f380053f0 tx=0x7f1f38004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff141e810770edb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38022040 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1f3801e640 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.750+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f38012030 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f3800a7d0 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 10 client.15132 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f95c06c060 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f38012030 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1f38026580 con 0x55f95c05dbb0 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 10 client.15132 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 20 client.15132 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c06e4d0 0x55f95c06f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 4 client.15132 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:55.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.751+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 --> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95bfc6600 con 0x55f95c06e4d0 2024-09-15T23:59:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.752+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c06e4d0 0x55f95c06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.752+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c06e4d0 0x55f95c06f940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa10e3226d20221d server_cookie=daaec98f27ae22c9 in_seq=0 out_seq=0 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.752+0000 7f1f1f7fe640 10 client.15132 ms_handle_connect on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/4064616549 <== mds.3 v2:172.21.15.102:6836/4082772592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f95bfc6600 con 0x55f95c06e4d0 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f1f7fe640 10 client.15132 handle_command_reply: tid=0 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 client.15132 shutdown 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 msgr2=0x55f95c05ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f380053f0 tx=0x7f1f38004a70 comp rx=0 tx=0).stop 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 shutdown_connections 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c05dbb0 0x55f95c05ad20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c06e4d0 0x55f95c06f940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c05a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 --2- 172.21.15.31:0/4064616549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c0529d0 0x55f95c05d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 >> 172.21.15.31:0/4064616549 conn(0x55f95c04ef60 msgr2=0x55f95c054c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f1f7fe640 0 client.15132 ms_handle_reset on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 shutdown_connections 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 1 -- 172.21.15.31:0/4064616549 wait complete. 2024-09-15T23:59:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.753+0000 7f1f48b43740 20 client.15132 trim_cache size 0 max 16384 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.771+0000 7f1f48b43740 1 Processor -- start 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.771+0000 7f1f48b43740 1 -- start start 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c077c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06dc80 con 0x55f95c0529d0 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06e350 con 0x55f95c05dbb0 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f48b43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06ea20 con 0x55f95c0532f0 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40474/0 (socket says 172.21.15.31:40474) 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 -- 172.21.15.31:0/1803012442 learned_addr learned my addr 172.21.15.31:0/1803012442 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 -- 172.21.15.31:0/1803012442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 msgr2=0x55f95c06a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c06a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 -- 172.21.15.31:0/1803012442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 msgr2=0x55f95c077c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c077c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 -- 172.21.15.31:0/1803012442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95c05d0b0 con 0x55f95c0529d0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b910 tx=0x7f1f3000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8529d1fee4c8bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f25ffb640 1 -- 172.21.15.31:0/1803012442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30022070 con 0x55f95c0529d0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f25ffb640 1 -- 172.21.15.31:0/1803012442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30003d50 con 0x55f95c0529d0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.772+0000 7f1f25ffb640 1 -- 172.21.15.31:0/1803012442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30015070 con 0x55f95c0529d0 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- 172.21.15.31:0/1803012442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 msgr2=0x55f95c06de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b910 tx=0x7f1f3000bd10 comp rx=0 tx=0).stop 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- 172.21.15.31:0/1803012442 shutdown_connections 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c077c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0532f0 0x55f95c06a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1803012442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0529d0 0x55f95c06de00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- 172.21.15.31:0/1803012442 >> 172.21.15.31:0/1803012442 conn(0x55f95c04ef60 msgr2=0x55f95c049ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- 172.21.15.31:0/1803012442 shutdown_connections 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- 172.21.15.31:0/1803012442 wait complete. 2024-09-15T23:59:55.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.773+0000 7f1f48b43740 1 -- start start 2024-09-15T23:59:55.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 Processor -- start 2024-09-15T23:59:55.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0529d0 0x55f95c072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06f7c0 con 0x55f95c0532f0 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06f0f0 con 0x55f95c05dbb0 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f95c06fe90 con 0x55f95c0529d0 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f48b43740 10 client.0 _ll_register_callbacks cb 0x7f1f38012720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40476/0 (socket says 172.21.15.31:40476) 2024-09-15T23:59:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1391253770 learned_addr learned my addr 172.21.15.31:0/1391253770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0529d0 0x55f95c072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f44d07640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0529d0 msgr2=0x55f95c072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0529d0 0x55f95c072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1391253770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 msgr2=0x55f95c073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.774+0000 7f1f3f7fe640 1 -- 172.21.15.31:0/1391253770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95c0d6bd0 con 0x55f95c0532f0 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f3f7fe640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f340bc320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe9ed268e21315 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34049040 con 0x55f95c0532f0 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1f34003c50 con 0x55f95c0532f0 2024-09-15T23:59:55.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f30003d50 con 0x55f95c0532f0 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f340bc720 con 0x55f95c0532f0 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 10 client.15138 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f95c0d7df0 con 0x55f95c0532f0 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30003d50 con 0x55f95c0532f0 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1f340b8d90 con 0x55f95c0532f0 2024-09-15T23:59:55.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 10 client.15138 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 20 client.15138 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c0dca20 0x55f95c0db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 4 client.15138 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.775+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 --> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] -- command(tid 0: {"prefix": "status"}) -- 0x55f95bfc0b10 con 0x55f95c0dca20 2024-09-15T23:59:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c0dca20 0x55f95c0db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f3ffff640 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c0dca20 0x55f95c0db730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=561c37a0d4a0d677 server_cookie=bbfd093d52e1f24c in_seq=0 out_seq=0 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f1f7fe640 10 client.15138 ms_handle_connect on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f1f7fe640 1 -- 172.21.15.31:0/1391253770 <== mds.3 v2:172.21.15.102:6836/4082772592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f95bfc0b10 con 0x55f95c0dca20 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f1f7fe640 10 client.15138 handle_command_reply: tid=0 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 client.15138 shutdown 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 msgr2=0x55f95c073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f340bc320 comp rx=0 tx=0).stop 2024-09-15T23:59:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 shutdown_connections 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f95c05dbb0 0x55f95c073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.776+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f95c0532f0 0x55f95c073100 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:6836/4082772592,v1:172.21.15.102:6837/4082772592] conn(0x55f95c0dca20 0x55f95c0db730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 1 --2- 172.21.15.31:0/1391253770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95c0529d0 0x55f95c072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 >> 172.21.15.31:0/1391253770 conn(0x55f95c04ef60 msgr2=0x55f95c04d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f1f7fe640 0 client.15138 ms_handle_reset on v2:172.21.15.102:6836/4082772592 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 shutdown_connections 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 1 -- 172.21.15.31:0/1391253770 wait complete. 2024-09-15T23:59:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f48b43740 20 client.15138 trim_cache size 0 max 16384 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24457, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 93.496007728999999, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:55.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 128.89794074599999 2024-09-15T23:59:55.790 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 msgr2=0x7f1f280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 0x7f1f280a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f38020000 tx=0x7f1f3801f000 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 msgr2=0x7f1f401988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005940 tx=0x7f1f34004270 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 shutdown_connections 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f1f280a3240 0x7f1f280a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f40106f10 0x7f1f4019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f40102790 0x7f1f401988e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 --2- 172.21.15.31:0/1737217179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f40101de0 0x7f1f401983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 >> 172.21.15.31:0/1737217179 conn(0x7f1f400fd8c0 msgr2=0x7f1f400fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 shutdown_connections 2024-09-15T23:59:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:55.777+0000 7f1f4676a640 1 -- 172.21.15.31:0/1737217179 wait complete. 2024-09-15T23:59:56.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:56 smithi081 ceph-mon[35060]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:56.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:56 smithi031 ceph-mon[31203]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:56.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:56 smithi102 ceph-mon[34458]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T23:59:56.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T23:59:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 -- 172.21.15.31:0/3752707350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4106f10 msgr2=0x7fcda410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 --2- 172.21.15.31:0/3752707350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4106f10 0x7fcda410ea90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009ca0 tx=0x7fcd9c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 -- 172.21.15.31:0/3752707350 shutdown_connections 2024-09-15T23:59:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 --2- 172.21.15.31:0/3752707350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4106f10 0x7fcda410ea90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 --2- 172.21.15.31:0/3752707350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4102790 0x7fcda41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 --2- 172.21.15.31:0/3752707350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4101de0 0x7fcda41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 -- 172.21.15.31:0/3752707350 >> 172.21.15.31:0/3752707350 conn(0x7fcda40fd8c0 msgr2=0x7fcda40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 -- 172.21.15.31:0/3752707350 shutdown_connections 2024-09-15T23:59:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.367+0000 7fcda9388640 1 -- 172.21.15.31:0/3752707350 wait complete. 2024-09-15T23:59:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 Processor -- start 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 -- start start 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4102790 0x7fcda4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 0x7fcda4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4111e20 con 0x7fcda4101de0 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4111ca0 con 0x7fcda4102790 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.368+0000 7fcda9388640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4111fa0 con 0x7fcda4106f10 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40486/0 (socket says 172.21.15.31:40486) 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 -- 172.21.15.31:0/2648181189 learned_addr learned my addr 172.21.15.31:0/2648181189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda37fe640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 0x7fcda4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 -- 172.21.15.31:0/2648181189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 msgr2=0x7fcda4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda27fc640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4102790 0x7fcda4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 0x7fcda4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 -- 172.21.15.31:0/2648181189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4102790 msgr2=0x7fcda4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4102790 0x7fcda4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 -- 172.21.15.31:0/2648181189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda41a6660 con 0x7fcda4101de0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda37fe640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 0x7fcda4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c920 tx=0x7fcd9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcd8bfff640 1 -- 172.21.15.31:0/2648181189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94007c70 con 0x7fcda4101de0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41a6950 con 0x7fcda4101de0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcd8bfff640 1 -- 172.21.15.31:0/2648181189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd94007e10 con 0x7fcda4101de0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcd8bfff640 1 -- 172.21.15.31:0/2648181189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94018820 con 0x7fcda4101de0 2024-09-15T23:59:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.369+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41a6e90 con 0x7fcda4101de0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.372+0000 7fcd8bfff640 1 -- 172.21.15.31:0/2648181189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcd940189c0 con 0x7fcda4101de0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.372+0000 7fcdab761740 1 Processor -- start 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.372+0000 7fcdab761740 1 -- start start 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 0x5653ba957670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba959fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5653ba8b3090 con 0x5653ba94c9d0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5653ba85aa70 con 0x5653ba94d2f0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcdab761740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653ba8af050 con 0x5653ba957bb0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 0x5653ba957670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40490/0 (socket says 172.21.15.31:40490) 2024-09-15T23:59:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 -- 172.21.15.31:0/4213375772 learned_addr learned my addr 172.21.15.31:0/4213375772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda27fc640 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba959fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcd8bfff640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 0x7fcd8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 -- 172.21.15.31:0/4213375772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 msgr2=0x5653ba959fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba959fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 -- 172.21.15.31:0/4213375772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 msgr2=0x5653ba957670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 0x5653ba957670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda37fe640 1 -- 172.21.15.31:0/4213375772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ba867130 con 0x5653ba94c9d0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 0x5653ba957670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.373+0000 7fcd8bfff640 1 -- 172.21.15.31:0/2648181189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd940cf9c0 con 0x7fcda4101de0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.374+0000 7fcda37fe640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 0x7fcd8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.374+0000 7fcda37fe640 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c031c70 tx=0x7fcd9c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9955f85a50f2827d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.374+0000 7fcd897fa640 1 -- 172.21.15.31:0/4213375772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c031750 con 0x5653ba94c9d0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.374+0000 7fcd897fa640 1 -- 172.21.15.31:0/4213375772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9c036070 con 0x5653ba94c9d0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.374+0000 7fcd897fa640 1 -- 172.21.15.31:0/4213375772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c030da0 con 0x5653ba94c9d0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- 172.21.15.31:0/4213375772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 msgr2=0x5653ba94cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcda37fe640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 0x7fcd8c0a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c04b000 tx=0x7fcd9c048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c031c70 tx=0x7fcd9c031ca0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- 172.21.15.31:0/4213375772 shutdown_connections 2024-09-15T23:59:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba959fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94d2f0 0x5653ba957670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- 172.21.15.31:0/4213375772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba94cdb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- 172.21.15.31:0/4213375772 >> 172.21.15.31:0/4213375772 conn(0x5653ba948f60 msgr2=0x5653ba949340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- 172.21.15.31:0/4213375772 shutdown_connections 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- 172.21.15.31:0/4213375772 wait complete. 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- start start 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 Processor -- start 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba954750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba954c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5653ba95a600 con 0x5653ba94c9d0 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5653ba86df20 con 0x5653ba957bb0 2024-09-15T23:59:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.375+0000 7fcdab761740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653ba95a780 con 0x5653ba94d2f0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcdab761740 10 client.0 _ll_register_callbacks cb 0x5653ba837030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40506/0 (socket says 172.21.15.31:40506) 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 -- 172.21.15.31:0/3572431386 learned_addr learned my addr 172.21.15.31:0/3572431386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda27fc640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba954750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda37fe640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba954c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 -- 172.21.15.31:0/3572431386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 msgr2=0x5653ba954750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba954750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 -- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 msgr2=0x5653ba954c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba954c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 -- 172.21.15.31:0/3572431386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ba6242b0 con 0x5653ba94c9d0 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda27fc640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba954750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda37fe640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba954c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c8f0 tx=0x7fcd940984f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391c260ee99d6b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcd86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9400fb00 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd940987a0 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9c036070 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd9400de20 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcdab761740 10 client.15150 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.376+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653ba965f30 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9c036070 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd940a4910 con 0x5653ba94c9d0 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 10 client.15150 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 20 client.15150 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba969a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 4 client.15150 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 --> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ba8c0600 con 0x5653ba9685c0 2024-09-15T23:59:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.377+0000 7fcda27fc640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba969a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.378+0000 7fcda27fc640 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba969a30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a106d3cb8a5d0db0 server_cookie=c7ea2b549de0217a in_seq=0 out_seq=0 2024-09-15T23:59:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.378+0000 7fcd86ff5640 10 client.15150 ms_handle_connect on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.378+0000 7fcd86ff5640 1 -- 172.21.15.31:0/3572431386 <== mds.2 v2:172.21.15.81:6836/3184710981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653ba8c0600 con 0x5653ba9685c0 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.378+0000 7fcd86ff5640 10 client.15150 handle_command_reply: tid=0 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.378+0000 7fcdab761740 1 client.15150 shutdown 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 msgr2=0x5653ba957400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c8f0 tx=0x7fcd940984f0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 shutdown_connections 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba969a30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba954c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcd86ff5640 0 client.15150 ms_handle_reset on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba954750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 --2- 172.21.15.31:0/3572431386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba957400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 >> 172.21.15.31:0/3572431386 conn(0x5653ba948f60 msgr2=0x5653ba94ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 shutdown_connections 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 1 -- 172.21.15.31:0/3572431386 wait complete. 2024-09-15T23:59:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.379+0000 7fcdab761740 20 client.15150 trim_cache size 0 max 16384 2024-09-15T23:59:57.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 Processor -- start 2024-09-15T23:59:57.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 -- start start 2024-09-15T23:59:57.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94c9d0 0x5653ba969a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba9519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5653ba967d70 con 0x5653ba94d2f0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5653ba968440 con 0x5653ba94c9d0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcdab761740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653ba968b10 con 0x5653ba957bb0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40508/0 (socket says 172.21.15.31:40508) 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 -- 172.21.15.31:0/2946859059 learned_addr learned my addr 172.21.15.31:0/2946859059 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 -- 172.21.15.31:0/2946859059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 msgr2=0x5653ba9519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba9519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 -- 172.21.15.31:0/2946859059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94c9d0 msgr2=0x5653ba969a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94c9d0 0x5653ba969a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 -- 172.21.15.31:0/2946859059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ba957770 con 0x5653ba94d2f0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.397+0000 7fcda27fc640 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5653ba94a2e0 tx=0x7fcd9800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d34af9b6741402 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcd88ff9640 1 -- 172.21.15.31:0/2946859059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9801c5b0 con 0x5653ba94d2f0 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcd88ff9640 1 -- 172.21.15.31:0/2946859059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9801cc00 con 0x5653ba94d2f0 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcd88ff9640 1 -- 172.21.15.31:0/2946859059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9801bde0 con 0x5653ba94d2f0 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 -- 172.21.15.31:0/2946859059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 msgr2=0x5653ba954010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5653ba94a2e0 tx=0x7fcd9800f7f0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 -- 172.21.15.31:0/2946859059 shutdown_connections 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba957bb0 0x5653ba9519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94d2f0 0x5653ba954010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 --2- 172.21.15.31:0/2946859059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba94c9d0 0x5653ba969a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 -- 172.21.15.31:0/2946859059 >> 172.21.15.31:0/2946859059 conn(0x5653ba948f60 msgr2=0x5653ba954710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 -- 172.21.15.31:0/2946859059 shutdown_connections 2024-09-15T23:59:57.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.398+0000 7fcdab761740 1 -- 172.21.15.31:0/2946859059 wait complete. 2024-09-15T23:59:57.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 -- start start 2024-09-15T23:59:57.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 Processor -- start 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba96d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba9d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5653ba9698b0 con 0x5653ba94c9d0 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5653ba9691e0 con 0x5653ba957bb0 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653ba969f80 con 0x5653ba94d2f0 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcdab761740 10 client.0 _ll_register_callbacks cb 0x7fcd9400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40522/0 (socket says 172.21.15.31:40522) 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 -- 172.21.15.31:0/441126422 learned_addr learned my addr 172.21.15.31:0/441126422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 -- 172.21.15.31:0/441126422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 msgr2=0x5653ba96d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba96d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 -- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 msgr2=0x5653ba9d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba9d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 -- 172.21.15.31:0/441126422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ba9d1200 con 0x5653ba94c9d0 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcda2ffd640 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd940a4870 tx=0x7fcd94007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa596e3ca146e791 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.399+0000 7fcd86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9409ba90 con 0x5653ba94c9d0 2024-09-15T23:59:57.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd9409b580 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9801cc00 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd9400fb00 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 10 client.15156 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653ba9d2480 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9801cc00 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd940a1670 con 0x5653ba94c9d0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 10 client.15156 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 20 client.15156 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba9d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 4 client.15156 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.400+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 --> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] -- command(tid 0: {"prefix": "status"}) -- 0x5653ba8bab10 con 0x5653ba9685c0 2024-09-15T23:59:57.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcda27fc640 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba9d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcda27fc640 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba9d5e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96058c84b48512e0 server_cookie=eea7e019e437dc74 in_seq=0 out_seq=0 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcd86ff5640 10 client.15156 ms_handle_connect on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcd86ff5640 1 -- 172.21.15.31:0/441126422 <== mds.2 v2:172.21.15.81:6836/3184710981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5653ba8bab10 con 0x5653ba9685c0 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcd86ff5640 10 client.15156 handle_command_reply: tid=0 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcdab761740 1 client.15156 shutdown 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 msgr2=0x5653ba96d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.401+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd940a4870 tx=0x7fcd94007730 comp rx=0 tx=0).stop 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 shutdown_connections 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:6836/3184710981,v1:172.21.15.81:6837/3184710981] conn(0x5653ba9685c0 0x5653ba9d5e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653ba957bb0 0x5653ba9d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653ba94d2f0 0x5653ba96d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 --2- 172.21.15.31:0/441126422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5653ba94c9d0 0x5653ba96d0e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 >> 172.21.15.31:0/441126422 conn(0x5653ba948f60 msgr2=0x5653ba9475c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcd86ff5640 0 client.15156 ms_handle_reset on v2:172.21.15.81:6836/3184710981 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 shutdown_connections 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 1 -- 172.21.15.31:0/441126422 wait complete. 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcdab761740 20 client.15156 trim_cache size 0 max 16384 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24461, 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:57.409 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 96.127574369000001, 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 127.81546225699999 2024-09-15T23:59:57.410 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 msgr2=0x7fcd8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 0x7fcd8c0a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c04b000 tx=0x7fcd9c048000 comp rx=0 tx=0).stop 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 msgr2=0x7fcda40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c920 tx=0x7fcd9400cde0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 shutdown_connections 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fcd8c0a3240 0x7fcd8c0a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda4106f10 0x7fcda4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcda4102790 0x7fcda4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 --2- 172.21.15.31:0/2648181189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4101de0 0x7fcda40744f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 >> 172.21.15.31:0/2648181189 conn(0x7fcda40fd8c0 msgr2=0x7fcda40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.402+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 shutdown_connections 2024-09-15T23:59:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:57.403+0000 7fcda9388640 1 -- 172.21.15.31:0/2648181189 wait complete. 2024-09-15T23:59:58.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T23:59:58.455 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:59:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 15 23:59:58 smithi031 ceph-mon[31203]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-15T23:59:58.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 15 23:59:58 smithi081 ceph-mon[35060]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-15T23:59:58.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 15 23:59:58 smithi102 ceph-mon[34458]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-15T23:59:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 -- 172.21.15.31:0/2509489929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4106f10 msgr2=0x7ff6c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/2509489929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4106f10 0x7ff6c410ea90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009ca0 tx=0x7ff6bc02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 -- 172.21.15.31:0/2509489929 shutdown_connections 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/2509489929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4106f10 0x7ff6c410ea90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/2509489929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4102790 0x7ff6c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/2509489929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4101de0 0x7ff6c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 -- 172.21.15.31:0/2509489929 >> 172.21.15.31:0/2509489929 conn(0x7ff6c40fd8c0 msgr2=0x7ff6c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 -- 172.21.15.31:0/2509489929 shutdown_connections 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.958+0000 7ff6cbd41640 1 -- 172.21.15.31:0/2509489929 wait complete. 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 Processor -- start 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 -- start start 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 0x7ff6c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4106f10 0x7ff6c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4111e40 con 0x7ff6c4101de0 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4111cc0 con 0x7ff6c4106f10 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6cbd41640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4111fc0 con 0x7ff6c4102790 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40552/0 (socket says 172.21.15.31:40552) 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 -- 172.21.15.31:0/405206485 learned_addr learned my addr 172.21.15.31:0/405206485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4106f10 0x7ff6c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 0x7ff6c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 -- 172.21.15.31:0/405206485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 msgr2=0x7ff6c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 0x7ff6c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 -- 172.21.15.31:0/405206485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4106f10 msgr2=0x7ff6c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4106f10 0x7ff6c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.959+0000 7ff6c9add640 1 -- 172.21.15.31:0/405206485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41a67e0 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 0x7ff6c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6c9add640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000b4a0 tx=0x7ff6c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6baffd640 1 -- 172.21.15.31:0/405206485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0014070 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6baffd640 1 -- 172.21.15.31:0/405206485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c0010ac0 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a6ad0 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6baffd640 1 -- 172.21.15.31:0/405206485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0017c30 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.960+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a6fc0 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6baffd640 1 -- 172.21.15.31:0/405206485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff6c0017070 con 0x7ff6c4101de0 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 Processor -- start 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 -- start start 2024-09-15T23:59:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2a96090 con 0x55cbb2b302f0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2a3da70 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.963+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2a92050 con 0x55cbb2b3abb0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40562/0 (socket says 172.21.15.31:40562) 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6c9add640 1 -- 172.21.15.31:0/451615516 learned_addr learned my addr 172.21.15.31:0/451615516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6baffd640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 0x7ff6b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 -- 172.21.15.31:0/451615516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 msgr2=0x55cbb2b3cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b3cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 -- 172.21.15.31:0/451615516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 msgr2=0x55cbb2b3a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 -- 172.21.15.31:0/451615516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb2a4a130 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6c9add640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6baffd640 1 -- 172.21.15.31:0/405206485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff6c00cec60 con 0x7ff6c4101de0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 0x7ff6b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.964+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b2fdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc031040 tx=0x7ff6bc031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6d3dc5fc3195b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 0x7ff6b00a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc045000 tx=0x7ff6bc047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6affff640 1 -- 172.21.15.31:0/451615516 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc003cb0 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6affff640 1 -- 172.21.15.31:0/451615516 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6bc036040 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6affff640 1 -- 172.21.15.31:0/451615516 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc030930 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 -- 172.21.15.31:0/451615516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 msgr2=0x55cbb2b2fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b2fdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc031040 tx=0x7ff6bc031b10 comp rx=0 tx=0).stop 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 -- 172.21.15.31:0/451615516 shutdown_connections 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b3cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b3a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/451615516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b2fdb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.965+0000 7ff6ce11a740 1 -- 172.21.15.31:0/451615516 >> 172.21.15.31:0/451615516 conn(0x55cbb2b2bf60 msgr2=0x55cbb2b2c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- 172.21.15.31:0/451615516 shutdown_connections 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- 172.21.15.31:0/451615516 wait complete. 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- start start 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 Processor -- start 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b302f0 0x55cbb2b377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b37d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b3d600 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2a50f20 con 0x55cbb2b3abb0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b3d780 con 0x55cbb2b302f0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6ce11a740 10 client.0 _ll_register_callbacks cb 0x55cbb2a1a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.966+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40564/0 (socket says 172.21.15.31:40564) 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 -- 172.21.15.31:0/2022552186 learned_addr learned my addr 172.21.15.31:0/2022552186 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b302f0 0x55cbb2b377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b37d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 -- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b302f0 msgr2=0x55cbb2b377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b302f0 0x55cbb2b377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 -- 172.21.15.31:0/2022552186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 msgr2=0x55cbb2b37d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b37d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 -- 172.21.15.31:0/2022552186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb28072b0 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00a3b10 tx=0x7ff6c009b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926a352fabed0ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c001e070 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff6c0097a40 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6bc036040 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6c009bad0 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ce11a740 10 client.15168 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:58.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbb2b48f80 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.967+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6bc036040 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff6c0010c70 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 10 client.15168 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 20 client.15168 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2b4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 4 client.15168 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 --> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb2aa3600 con 0x55cbb2b4b640 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2b4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2b4cab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11731275edb60ba9 server_cookie=859ed867e8a9d905 in_seq=0 out_seq=0 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.968+0000 7ff6ad7fa640 10 client.15168 ms_handle_connect on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:58.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2022552186 <== mds.1 v2:172.21.15.102:6838/3686122257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbb2aa3600 con 0x55cbb2b4b640 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ad7fa640 10 client.15168 handle_command_reply: tid=0 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 client.15168 shutdown 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 msgr2=0x55cbb2b3a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00a3b10 tx=0x7ff6c009b220 comp rx=0 tx=0).stop 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 shutdown_connections 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2b4cab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b37d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b302f0 0x55cbb2b377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ad7fa640 0 client.15168 ms_handle_reset on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:58.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.969+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2022552186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b3a480 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.970+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 >> 172.21.15.31:0/2022552186 conn(0x55cbb2b2bf60 msgr2=0x55cbb2b31c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.970+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 shutdown_connections 2024-09-15T23:59:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.970+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2022552186 wait complete. 2024-09-15T23:59:58.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.970+0000 7ff6ce11a740 20 client.15168 trim_cache size 0 max 16384 2024-09-15T23:59:58.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 Processor -- start 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 -- start start 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4adf0 con 0x55cbb2b302f0 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4b4c0 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4bb90 con 0x55cbb2b3abb0 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6c92dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6c92dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40568/0 (socket says 172.21.15.31:40568) 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2855061193 learned_addr learned my addr 172.21.15.31:0/2855061193 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.988+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2855061193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 msgr2=0x55cbb2b360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2855061193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 msgr2=0x55cbb2b4cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2855061193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb2b3d2e0 con 0x55cbb2b302f0 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4005fd0 tx=0x7ff6b400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83870cfc20e2858 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6af7fe640 1 -- 172.21.15.31:0/2855061193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4005220 con 0x55cbb2b302f0 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6af7fe640 1 -- 172.21.15.31:0/2855061193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6b4004c50 con 0x55cbb2b302f0 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.989+0000 7ff6af7fe640 1 -- 172.21.15.31:0/2855061193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b401c960 con 0x55cbb2b302f0 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2855061193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 msgr2=0x55cbb2b37090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4005fd0 tx=0x7ff6b400f7f0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2855061193 shutdown_connections 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b3abb0 0x55cbb2b360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b37090 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2855061193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2855061193 >> 172.21.15.31:0/2855061193 conn(0x55cbb2b2bf60 msgr2=0x55cbb2b37790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2855061193 shutdown_connections 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2855061193 wait complete. 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 -- start start 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.990+0000 7ff6ce11a740 1 Processor -- start 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2bb3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4c930 con 0x55cbb2b302f0 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4c260 con 0x55cbb2b3abb0 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2b4d000 con 0x55cbb2b2f9d0 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ce11a740 10 client.0 _ll_register_callbacks cb 0x7ff6c000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40578/0 (socket says 172.21.15.31:40578) 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2816730616 learned_addr learned my addr 172.21.15.31:0/2816730616 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ca2de640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2bb3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b2f9d0 msgr2=0x55cbb2b4ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2816730616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 msgr2=0x55cbb2bb3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2bb3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 -- 172.21.15.31:0/2816730616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb2b4ebc0 con 0x55cbb2b302f0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6c92dc640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400bfd0 tx=0x7ff6b4004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258449db0a091416 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.991+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b401c8f0 con 0x55cbb2b302f0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff6b401ca90 con 0x55cbb2b302f0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b4004c50 con 0x55cbb2b302f0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6b4039050 con 0x55cbb2b302f0 2024-09-15T23:59:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbb2bb5380 con 0x55cbb2b302f0 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6b4004c50 con 0x55cbb2b302f0 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff6b402e020 con 0x55cbb2b302f0 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 20 client.15174 populate_metadata read hostname 'smithi031' 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2bb89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 4 client.15174 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.992+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 --> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] -- command(tid 0: {"prefix": "status"}) -- 0x55cbb2a9db10 con 0x55cbb2b4b640 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2bb89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6c9add640 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2bb89b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=478810b54de2c34a server_cookie=f6813685e04423b9 in_seq=0 out_seq=0 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ad7fa640 10 client.15174 ms_handle_connect on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ad7fa640 1 -- 172.21.15.31:0/2816730616 <== mds.1 v2:172.21.15.102:6838/3686122257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbb2a9db10 con 0x55cbb2b4b640 2024-09-15T23:59:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ad7fa640 10 client.15174 handle_command_reply: tid=0 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 client.15174 shutdown 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 msgr2=0x55cbb2b4e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400bfd0 tx=0x7ff6b4004960 comp rx=0 tx=0).stop 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 shutdown_connections 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:6838/3686122257,v1:172.21.15.102:6839/3686122257] conn(0x55cbb2b4b640 0x55cbb2bb89b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbb2b3abb0 0x55cbb2bb3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbb2b302f0 0x55cbb2b4e2f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 --2- 172.21.15.31:0/2816730616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbb2b2f9d0 0x55cbb2b4ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ad7fa640 0 client.15174 ms_handle_reset on v2:172.21.15.102:6838/3686122257 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.993+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 >> 172.21.15.31:0/2816730616 conn(0x55cbb2b2bf60 msgr2=0x55cbb2b37790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 shutdown_connections 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6ce11a740 1 -- 172.21.15.31:0/2816730616 wait complete. 2024-09-15T23:59:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6ce11a740 20 client.15174 trim_cache size 0 max 16384 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24478, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 98.724542827999997, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-15T23:59:59.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 126.783533971 2024-09-15T23:59:59.003 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-15T23:59:59.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 msgr2=0x7ff6b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 0x7ff6b00a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc045000 tx=0x7ff6bc047000 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 msgr2=0x7ff6c40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000b4a0 tx=0x7ff6c000b960 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 shutdown_connections 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff6b00a3240 0x7ff6b00a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c4106f10 0x7ff6c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c4102790 0x7ff6c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 --2- 172.21.15.31:0/405206485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c4101de0 0x7ff6c40744f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 >> 172.21.15.31:0/405206485 conn(0x7ff6c40fd8c0 msgr2=0x7ff6c40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 shutdown_connections 2024-09-15T23:59:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-15T23:59:58.994+0000 7ff6cbd41640 1 -- 172.21.15.31:0/405206485 wait complete. 2024-09-15T23:59:59.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:00:00.098 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:00:00.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:00 smithi031 ceph-mon[31203]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:00:00.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:00 smithi031 ceph-mon[31203]: overall HEALTH_OK 2024-09-16T00:00:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:00 smithi081 ceph-mon[35060]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:00:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:00 smithi081 ceph-mon[35060]: overall HEALTH_OK 2024-09-16T00:00:00.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:00 smithi102 ceph-mon[34458]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:00:00.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:00 smithi102 ceph-mon[34458]: overall HEALTH_OK 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.705+0000 7f5ded393640 1 -- 172.21.15.31:0/150470155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8102790 msgr2=0x7f5de81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.705+0000 7f5ded393640 1 --2- 172.21.15.31:0/150470155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8102790 0x7f5de81067e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009900 tx=0x7f5ddc02a410 comp rx=0 tx=0).stop 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.705+0000 7f5ded393640 1 -- 172.21.15.31:0/150470155 shutdown_connections 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 --2- 172.21.15.31:0/150470155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 0x7f5de810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 --2- 172.21.15.31:0/150470155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8102790 0x7f5de81067e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 --2- 172.21.15.31:0/150470155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8101de0 0x7f5de81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 -- 172.21.15.31:0/150470155 >> 172.21.15.31:0/150470155 conn(0x7f5de80fd8c0 msgr2=0x7f5de80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 -- 172.21.15.31:0/150470155 shutdown_connections 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 -- 172.21.15.31:0/150470155 wait complete. 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 Processor -- start 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 -- start start 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.706+0000 7f5ded393640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8102790 0x7f5de81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 0x7f5de819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8111e70 con 0x7f5de8101de0 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8111cf0 con 0x7f5de8102790 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8111ff0 con 0x7f5de8106f10 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40590/0 (socket says 172.21.15.31:40590) 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 -- 172.21.15.31:0/557408734 learned_addr learned my addr 172.21.15.31:0/557408734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de77fe640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 0x7f5de819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de67fc640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8102790 0x7f5de81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 -- 172.21.15.31:0/557408734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 msgr2=0x7f5de819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 0x7f5de819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 -- 172.21.15.31:0/557408734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8102790 msgr2=0x7f5de81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8102790 0x7f5de81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 -- 172.21.15.31:0/557408734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de819d320 con 0x7f5de8101de0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d950 tx=0x7f5dd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5dcffff640 1 -- 172.21.15.31:0/557408734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8017030 con 0x7f5de8101de0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5dcffff640 1 -- 172.21.15.31:0/557408734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd8004020 con 0x7f5de8101de0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de819d610 con 0x7f5de8101de0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5dcffff640 1 -- 172.21.15.31:0/557408734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80047a0 con 0x7f5de8101de0 2024-09-16T00:00:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.707+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81a4ef0 con 0x7f5de8101de0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5dcffff640 1 -- 172.21.15.31:0/557408734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5dd80041d0 con 0x7f5de8101de0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 Processor -- start 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 -- start start 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5dcffff640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb924238b0 0x55bb92423c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924241d0 0x55bb92432a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb9238a090 con 0x55bb92432fc0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb92331a70 con 0x55bb924238b0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5def76c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb92386050 con 0x55bb924241d0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.710+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5dcffff640 1 -- 172.21.15.31:0/557408734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5dd80cf1b0 con 0x7f5de8101de0 2024-09-16T00:00:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40592/0 (socket says 172.21.15.31:40592) 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 -- 172.21.15.31:0/52388019 learned_addr learned my addr 172.21.15.31:0/52388019 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de67fc640 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924241d0 0x55bb92432a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb924238b0 0x55bb92423c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 -- 172.21.15.31:0/52388019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924241d0 msgr2=0x55bb92432a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924241d0 0x55bb92432a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 -- 172.21.15.31:0/52388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb924238b0 msgr2=0x55bb92423c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb924238b0 0x55bb92423c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 -- 172.21.15.31:0/52388019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9233e130 con 0x55bb92432fc0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5de0012d10 tx=0x7f5de0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852caf5ddca9ddac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5de77fe640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 0x7f5dd00a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5de0010c40 tx=0x7f5de0017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5dcd7fa640 1 -- 172.21.15.31:0/52388019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0023070 con 0x55bb92432fc0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5dcd7fa640 1 -- 172.21.15.31:0/52388019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de0009ce0 con 0x55bb92432fc0 2024-09-16T00:00:00.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.711+0000 7f5dcd7fa640 1 -- 172.21.15.31:0/52388019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de000de40 con 0x55bb92432fc0 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- 172.21.15.31:0/52388019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 msgr2=0x55bb924353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5de0012d10 tx=0x7f5de0010740 comp rx=0 tx=0).stop 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- 172.21.15.31:0/52388019 shutdown_connections 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb92432fc0 0x55bb924353b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924241d0 0x55bb92432a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 --2- 172.21.15.31:0/52388019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb924238b0 0x55bb92423c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- 172.21.15.31:0/52388019 >> 172.21.15.31:0/52388019 conn(0x55bb9241ff60 msgr2=0x55bb92420340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- 172.21.15.31:0/52388019 shutdown_connections 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- 172.21.15.31:0/52388019 wait complete. 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.712+0000 7f5def76c740 1 -- start start 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 Processor -- start 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb92432850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb924300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb92435b90 con 0x55bb924241d0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb92344f20 con 0x55bb92432fc0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb92435d10 con 0x55bb924238b0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5def76c740 10 client.0 _ll_register_callbacks cb 0x55bb9230e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40608/0 (socket says 172.21.15.31:40608) 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 -- 172.21.15.31:0/328104011 learned_addr learned my addr 172.21.15.31:0/328104011 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb92432850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de77fe640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb924300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 -- 172.21.15.31:0/328104011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 msgr2=0x55bb92432850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb92432850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 -- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 msgr2=0x55bb924300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb924300e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 -- 172.21.15.31:0/328104011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb920fb2b0 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5de67fc640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000c00 tx=0x7f5ddc004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cab92a135a27d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.713+0000 7f5dcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035040 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5ddc005510 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de0009ce0 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc004770 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 10 client.15186 fetch_fsmap learned FSMap version 27 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb924414e0 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5de0009ce0 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ddc003b50 con 0x55bb924241d0 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T00:00:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 20 client.15186 populate_metadata read hostname 'smithi031' 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb92444f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 4 client.15186 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.714+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 --> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb92397600 con 0x55bb92443ae0 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.715+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb92444f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.715+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb92444f50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d53ab144af3389 server_cookie=b5ae19a19b73d4bb in_seq=0 out_seq=0 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.715+0000 7f5dcaff5640 10 client.15186 ms_handle_connect on v2:172.21.15.81:6838/471643993 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5dcaff5640 1 -- 172.21.15.31:0/328104011 <== mds.0 v2:172.21.15.81:6838/471643993 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb92397600 con 0x55bb92443ae0 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5dcaff5640 10 client.15186 handle_command_reply: tid=0 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 client.15186 shutdown 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 msgr2=0x55bb9242fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000c00 tx=0x7f5ddc004510 comp rx=0 tx=0).stop 2024-09-16T00:00:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 shutdown_connections 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb924300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242fba0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb92444f50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 --2- 172.21.15.31:0/328104011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb92432850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5dcaff5640 0 client.15186 ms_handle_reset on v2:172.21.15.81:6838/471643993 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 >> 172.21.15.31:0/328104011 conn(0x55bb9241ff60 msgr2=0x55bb92425cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 shutdown_connections 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 1 -- 172.21.15.31:0/328104011 wait complete. 2024-09-16T00:00:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.716+0000 7f5def76c740 20 client.15186 trim_cache size 0 max 16384 2024-09-16T00:00:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 Processor -- start 2024-09-16T00:00:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 -- start start 2024-09-16T00:00:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9242eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb9242d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb92443290 con 0x55bb924241d0 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb92443960 con 0x55bb92432fc0 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5def76c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb92444030 con 0x55bb924238b0 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40614/0 (socket says 172.21.15.31:40614) 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.734+0000 7f5de67fc640 1 -- 172.21.15.31:0/2506352037 learned_addr learned my addr 172.21.15.31:0/2506352037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 -- 172.21.15.31:0/2506352037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 msgr2=0x55bb9242eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9242eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 -- 172.21.15.31:0/2506352037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 msgr2=0x55bb9242d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb9242d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 -- 172.21.15.31:0/2506352037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb924210d0 con 0x55bb924241d0 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0315f0 tx=0x7f5ddc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a24957ca8c93e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5dccff9640 1 -- 172.21.15.31:0/2506352037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035070 con 0x55bb924241d0 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5dccff9640 1 -- 172.21.15.31:0/2506352037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040070 con 0x55bb924241d0 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5dccff9640 1 -- 172.21.15.31:0/2506352037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc004b70 con 0x55bb924241d0 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5def76c740 1 -- 172.21.15.31:0/2506352037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 msgr2=0x55bb9242ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.735+0000 7f5def76c740 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0315f0 tx=0x7f5ddc02a410 comp rx=0 tx=0).stop 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- 172.21.15.31:0/2506352037 shutdown_connections 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb9242d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb9242ce20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- 172.21.15.31:0/2506352037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9242eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- 172.21.15.31:0/2506352037 >> 172.21.15.31:0/2506352037 conn(0x55bb9241ff60 msgr2=0x55bb9242e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- 172.21.15.31:0/2506352037 shutdown_connections 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- 172.21.15.31:0/2506352037 wait complete. 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- start start 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 Processor -- start 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9244d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb92447180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bb92444dd0 con 0x55bb924241d0 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb92444700 con 0x55bb92432fc0 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb924454a0 con 0x55bb924238b0 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5def76c740 10 client.0 _ll_register_callbacks cb 0x7f5ddc0041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5de67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40628/0 (socket says 172.21.15.31:40628) 2024-09-16T00:00:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.736+0000 7f5de67fc640 1 -- 172.21.15.31:0/2544224551 learned_addr learned my addr 172.21.15.31:0/2544224551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9244d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de77fe640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb92447180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 -- 172.21.15.31:0/2544224551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 msgr2=0x55bb9244d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9244d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 -- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 msgr2=0x55bb92447180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb92447180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 -- 172.21.15.31:0/2544224551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9241f810 con 0x55bb924241d0 2024-09-16T00:00:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9244d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de77fe640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb92447180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5de67fc640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc02a930 tx=0x7f5ddc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7114544e170604c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5dcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035070 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5ddc031860 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc040070 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc005850 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5def76c740 10 client.15192 fetch_fsmap learned FSMap version 27 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.737+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb924ae1f0 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040070 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ddc045470 con 0x55bb924241d0 2024-09-16T00:00:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 20 client.15192 populate_metadata read hostname 'smithi031' 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb924b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 4 client.15192 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 --> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] -- command(tid 0: {"prefix": "status"}) -- 0x55bb92391b10 con 0x55bb92443ae0 2024-09-16T00:00:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.738+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb924b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5de6ffd640 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb924b1570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e833e8e9af349050 server_cookie=a864b6c036a3cd6b in_seq=0 out_seq=0 2024-09-16T00:00:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5dcaff5640 10 client.15192 ms_handle_connect on v2:172.21.15.81:6838/471643993 2024-09-16T00:00:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5dcaff5640 1 -- 172.21.15.31:0/2544224551 <== mds.0 v2:172.21.15.81:6838/471643993 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bb92391b10 con 0x55bb92443ae0 2024-09-16T00:00:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5dcaff5640 10 client.15192 handle_command_reply: tid=0 2024-09-16T00:00:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 client.15192 shutdown 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 msgr2=0x55bb92446c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc02a930 tx=0x7f5ddc004b40 comp rx=0 tx=0).stop 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 shutdown_connections 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb92432fc0 0x55bb92447180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb924241d0 0x55bb92446c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.81:6838/471643993,v1:172.21.15.81:6839/471643993] conn(0x55bb92443ae0 0x55bb924b1570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 --2- 172.21.15.31:0/2544224551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb924238b0 0x55bb9244d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5dcaff5640 0 client.15192 ms_handle_reset on v2:172.21.15.81:6838/471643993 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 >> 172.21.15.31:0/2544224551 conn(0x55bb9241ff60 msgr2=0x55bb9241f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 shutdown_connections 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 1 -- 172.21.15.31:0/2544224551 wait complete. 2024-09-16T00:00:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.739+0000 7f5def76c740 20 client.15192 trim_cache size 0 max 16384 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "21a99a0a-73bd-11ef-bceb-c7b262605968", 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24491, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 108.599095378, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 119.387667976 2024-09-16T00:00:00.758 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 msgr2=0x7f5dd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 0x7f5dd00a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5de0010c40 tx=0x7f5de0017000 comp rx=0 tx=0).stop 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 msgr2=0x7f5de8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d950 tx=0x7f5dd800de10 comp rx=0 tx=0).stop 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 shutdown_connections 2024-09-16T00:00:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7f5dd00a3240 0x7f5dd00a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de8106f10 0x7f5de819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5de8102790 0x7f5de81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 --2- 172.21.15.31:0/557408734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5de8101de0 0x7f5de8198370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 >> 172.21.15.31:0/557408734 conn(0x7f5de80fd8c0 msgr2=0x7f5de80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 shutdown_connections 2024-09-16T00:00:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:00.740+0000 7f5ded393640 1 -- 172.21.15.31:0/557408734 wait complete. 2024-09-16T00:00:01.571 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:00:01.581 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:00:01.581 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:00:01.581 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:00:01.581 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-16T00:00:01.582 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:00:01.582 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:00:01.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-16T00:00:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 -- 172.21.15.31:0/1408765617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100d60 msgr2=0x7ff2181011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 --2- 172.21.15.31:0/1408765617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100d60 0x7ff2181011c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009900 tx=0x7ff20c02a410 comp rx=0 tx=0).stop 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 -- 172.21.15.31:0/1408765617 shutdown_connections 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 --2- 172.21.15.31:0/1408765617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 0x7ff218101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 --2- 172.21.15.31:0/1408765617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100d60 0x7ff2181011c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 --2- 172.21.15.31:0/1408765617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100420 0x7ff218100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.688+0000 7ff220198640 1 -- 172.21.15.31:0/1408765617 >> 172.21.15.31:0/1408765617 conn(0x7ff2180fc1b0 msgr2=0x7ff2180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- 172.21.15.31:0/1408765617 shutdown_connections 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- 172.21.15.31:0/1408765617 wait complete. 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 Processor -- start 2024-09-16T00:00:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- start start 2024-09-16T00:00:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100d60 0x7ff21819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 0x7ff2181a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810f810 con 0x7ff218100420 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810f690 con 0x7ff218100d60 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff220198640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810f990 con 0x7ff218101700 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff21df34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff21df34640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53850/0 (socket says 172.21.15.31:53850) 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.689+0000 7ff21df34640 1 -- 172.21.15.31:0/327583963 learned_addr learned my addr 172.21.15.31:0/327583963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21d733640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100d60 0x7ff21819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21e735640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 0x7ff2181a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 -- 172.21.15.31:0/327583963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 msgr2=0x7ff2181a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 0x7ff2181a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 -- 172.21.15.31:0/327583963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100d60 msgr2=0x7ff21819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100d60 0x7ff21819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 -- 172.21.15.31:0/327583963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181a1a30 con 0x7ff218100420 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff21df34640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff20800c920 tx=0x7ff20800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208007c70 con 0x7ff218100420 2024-09-16T00:00:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff208007e10 con 0x7ff218100420 2024-09-16T00:00:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181a1d20 con 0x7ff218100420 2024-09-16T00:00:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208018820 con 0x7ff218100420 2024-09-16T00:00:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.690+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a2200 con 0x7ff218100420 2024-09-16T00:00:01.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.691+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21810b170 con 0x7ff218100420 2024-09-16T00:00:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.693+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff2080189c0 con 0x7ff218100420 2024-09-16T00:00:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.693+0000 7ff216ffd640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 0x7ff2000a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.693+0000 7ff21d733640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 0x7ff2000a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.693+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff2080cf940 con 0x7ff218100420 2024-09-16T00:00:01.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.694+0000 7ff21d733640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 0x7ff2000a5830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff21819b320 tx=0x7ff20c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.698+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff208014030 con 0x7ff218100420 2024-09-16T00:00:01.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:01.921+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ff218062e90 con 0x7ff218100420 2024-09-16T00:00:02.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.310+0000 7ff216ffd640 1 -- 172.21.15.31:0/327583963 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7ff20800de20 con 0x7ff218100420 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 msgr2=0x7ff2000a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 0x7ff2000a5830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff21819b320 tx=0x7ff20c0314a0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 msgr2=0x7ff218199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff20800c920 tx=0x7ff20800cde0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 shutdown_connections 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7ff2000a3370 0x7ff2000a5830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff218101700 0x7ff2181a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff218100d60 0x7ff21819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 --2- 172.21.15.31:0/327583963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218100420 0x7ff218199e10 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff20800c920 tx=0x7ff20800cde0 comp rx=0 tx=0).stop 2024-09-16T00:00:02.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 >> 172.21.15.31:0/327583963 conn(0x7ff2180fc1b0 msgr2=0x7ff2180fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 shutdown_connections 2024-09-16T00:00:02.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:02.313+0000 7ff220198640 1 -- 172.21.15.31:0/327583963 wait complete. 2024-09-16T00:00:02.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-mon[35060]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:00:02.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T00:00:02.331 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-16T00:00:02.358 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-16T00:00:02.359 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-16T00:00:02.359 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-16T00:00:02.359 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:00:02.359 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:02.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:02.359 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:00:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:02.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T00:00:02.447 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-09-16T00:00:02.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:00:02.449 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-16T00:00:02.449 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T00:00:02.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:00:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-mon[34458]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:00:02.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T00:00:02.622 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: ignoring --setuser ceph since I am not root 2024-09-16T00:00:02.622 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: ignoring --setgroup ceph since I am not root 2024-09-16T00:00:02.622 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-mgr[36269]: -- 172.21.15.102:0/2413771152 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55784d08d0e0 con 0x55784d06a800 2024-09-16T00:00:02.622 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:02.468+0000 7f5194528040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:00:02.622 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:02.526+0000 7f5194528040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: ignoring --setuser ceph since I am not root 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: ignoring --setgroup ceph since I am not root 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-mgr[36427]: -- 172.21.15.81:0/40335704 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4438870e0 con 0x55b443865400 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:02.467+0000 7f427e167040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:02.525+0000 7f427e167040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:02.678 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:02.676+0000 7f427e167040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-mon[31203]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setuser ceph since I am not root 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ignoring --setgroup ceph since I am not root 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:02.469+0000 7f6b49a90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:02.530+0000 7f6b49a90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:02.690 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:02.688+0000 7f6b49a90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:00:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:00:03.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:02.766+0000 7f6b49a90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:00:03.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:02.922+0000 7f6b49a90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:03.037 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:02.677+0000 7f5194528040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:00:03.038 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:02.746+0000 7f5194528040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:00:03.038 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:02.894+0000 7f5194528040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:03.038 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:03.036+0000 7f5194528040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:00:03.043 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:02.745+0000 7f427e167040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:00:03.043 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:02.897+0000 7f427e167040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:03.043 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:03.042+0000 7f427e167040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:00:03.313 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:03.078+0000 7f6b49a90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.483+0000 7fa48ab0c640 1 -- 172.21.15.31:0/3295524065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa484103ff0 msgr2=0x7fa4841043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.483+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/3295524065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa484103ff0 0x7fa4841043d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa478009900 tx=0x7fa47802a410 comp rx=0 tx=0).stop 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 -- 172.21.15.31:0/3295524065 shutdown_connections 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/3295524065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 0x7fa484110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/3295524065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa4841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/3295524065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa484103ff0 0x7fa4841043d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 -- 172.21.15.31:0/3295524065 >> 172.21.15.31:0/3295524065 conn(0x7fa4840ffab0 msgr2=0x7fa484101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 -- 172.21.15.31:0/3295524065 shutdown_connections 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 -- 172.21.15.31:0/3295524065 wait complete. 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 Processor -- start 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.484+0000 7fa48ab0c640 1 -- start start 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 0x7fa48419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 0x7fa48419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa484113ff0 con 0x7fa4841049a0 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa484113e70 con 0x7fa484109120 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa48ab0c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114170 con 0x7fa484103ff0 2024-09-16T00:00:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53880/0 (socket says 172.21.15.31:53880) 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 -- 172.21.15.31:0/1292207035 learned_addr learned my addr 172.21.15.31:0/1292207035 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa4888a8640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 0x7fa48419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa4890a9640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 0x7fa48419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 msgr2=0x7fa48419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 0x7fa48419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 msgr2=0x7fa48419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 0x7fa48419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 -- 172.21.15.31:0/1292207035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419f4c0 con 0x7fa4841049a0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa4888a8640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 0x7fa48419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.485+0000 7fa483fff640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e9f0 tx=0x7fa47000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.486+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000cd80 con 0x7fa4841049a0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.486+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419f7b0 con 0x7fa4841049a0 2024-09-16T00:00:03.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.486+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4700054d0 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.486+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470010490 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.486+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a7090 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.487+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4841043d0 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.488+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fa470005020 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.489+0000 7fa481ffb640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 0x7fa4740a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.489+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa4700cfb80 con 0x7fa4841049a0 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.490+0000 7fa4888a8640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 msgr2=0x7fa4740a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/797003742 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.490+0000 7fa4888a8640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 0x7fa4740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:00:03.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.495+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa47009c9f0 con 0x7fa4841049a0 2024-09-16T00:00:03.615 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T00:00:03.615 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-mon[34458]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-16T00:00:03.616 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:03.361+0000 7f5194528040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:00:03.616 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:03.422+0000 7f5194528040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:00:03.625 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T00:00:03.626 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-mon[35060]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-16T00:00:03.626 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:03.368+0000 7f427e167040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:00:03.626 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:03.432+0000 7f427e167040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:00:03.626 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:03.625+0000 7f427e167040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:00:03.634 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:03.411+0000 7f6b49a90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:00:03.634 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:03.476+0000 7f6b49a90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:00:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/327583963' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-16T00:00:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-mon[31203]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-16T00:00:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.691+0000 7fa4888a8640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 msgr2=0x7fa4740a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/797003742 2024-09-16T00:00:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.691+0000 7fa4888a8640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 0x7fa4740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:00:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.714+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa48410bd70 con 0x7fa4841049a0 2024-09-16T00:00:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.714+0000 7fa481ffb640 1 -- 172.21.15.31:0/1292207035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fa47009c2f0 con 0x7fa4841049a0 2024-09-16T00:00:03.715 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 msgr2=0x7fa4740a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 0x7fa4740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 msgr2=0x7fa48419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e9f0 tx=0x7fa47000edf0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 shutdown_connections 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:6800/797003742,v1:172.21.15.31:6801/797003742] conn(0x7fa4740a3290 0x7fa4740a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa484109120 0x7fa48419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4841049a0 0x7fa48419aa50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 --2- 172.21.15.31:0/1292207035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484103ff0 0x7fa48419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 >> 172.21.15.31:0/1292207035 conn(0x7fa4840ffab0 msgr2=0x7fa484100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 shutdown_connections 2024-09-16T00:00:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:03.716+0000 7fa48ab0c640 1 -- 172.21.15.31:0/1292207035 wait complete. 2024-09-16T00:00:03.870 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:03.614+0000 7f5194528040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:00:04.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:03.676+0000 7f6b49a90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:00:04.526 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:00:04.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:04.527 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-16T00:00:04.546 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:00:04.546 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:00:04.546 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:00:04.546 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-16T00:00:04.547 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42183sec preferred_lft 42183sec 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T00:00:04.548 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:00:04.549 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:00:04.549 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-16T00:00:04.549 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-16T00:00:04.549 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:00:04.549 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:00:04.549 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:00:04.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:00:04.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-16T00:00:04.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-16T00:00:04.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:00:04.550 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:00:04.592 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:04 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1292207035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:00:04.592 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:04.525+0000 7f427e167040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:00:04.603 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:04.535+0000 7f5194528040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:00:04.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:04 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1292207035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:00:04.616 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:04 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1292207035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:00:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:00:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:04.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:04.757 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:00:04.823 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-16T00:00:04.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:04.825 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-16T00:00:04.870 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:04.602+0000 7f5194528040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:00:04.890 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-16T00:00:04.890 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:00:04.890 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:00:04.890 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:00:04.890 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:00:04.891 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:00:04.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:00:04.990 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:04.615+0000 7f6b49a90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:00:04.991 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:04.689+0000 7f6b49a90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:00:05.057 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:04.591+0000 7f427e167040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:00:05.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:05.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:05.192 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:00:05.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:05.209 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-16T00:00:05.265 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:00:05.265 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:00:05.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:05.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:00:05.265 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:00:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:00:05.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:05.387 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:00:05.388 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:00:05.388 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:00:05.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:00:05.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:00:05.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:00:05.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:00:05.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:00:05.389 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:00:05.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:05.528 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:00:05.528 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T00:00:05.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-16T00:00:05.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:00:05.528 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T00:00:05.566 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.308+0000 7f5194528040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:00:05.566 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.425+0000 7f5194528040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:00:05.570 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.309+0000 7f427e167040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:00:05.570 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.426+0000 7f427e167040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:00:05.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:00:05.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:05.609 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-16T00:00:05.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:00:05.609 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:05.625 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:00:05.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:00:05.625 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:05.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-09-16T00:00:05.745 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:05.460+0000 7f6b49a90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:00:05.745 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:05.589+0000 7f6b49a90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:00:05.750 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-09-16T00:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-09-16T00:00:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/18782f8e2f502102cb1a5d248dc8da6b9fdae02e30926ec497ad50b000ff2992/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c5fba984cf18a7bf2d1f64ee2cda5bbeac0ac979101c7b8b46e568420d0a5140/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/95a6553b81a4e718eb37aae9489f0ea17a383d6d2deb7c04a704e1241524cef6/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/372c2f8b36c16df048d603fb7b5a937897ae330bea6e91c54e36dba03eb9a21a/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e4e7cabb51689eb4bb8dd0b237f4f2a7bdfdf79bd8e772372b2fe539193f420c/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b6ffe435bde5f2d3308c25c9d6449f80fec4c43ed1211892c08228d96a35557e/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/eebf208e408df8e642bc0a2912044e2ae5fedb056f473dcf31505a35cca07444/merged 2024-09-16T00:00:05.810 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/6fd3e5e5f8304de5adc7c67fd81c7f1d1f6e9bd7aa26ae2d6c90ff9437d309a2/merged 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/2226907b8c81f952bc1a40c4e5bac91bc987d5dc498feef5397d13eb1e2eaf31/merged 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/208c93059dc4a78a65f56d18419e6c595ca845f6f0378bb95dbc00e32b350c61/merged 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:05.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:05.811 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-09-16T00:00:05.842 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.565+0000 7f5194528040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:05.842 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.626+0000 7f5194528040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:00:05.843 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.773+0000 7f5194528040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:00:05.843 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.841+0000 7f5194528040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:00:05.850 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.569+0000 7f427e167040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:05.850 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.631+0000 7f427e167040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:00:05.850 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.781+0000 7f427e167040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:00:05.850 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.849+0000 7f427e167040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:00:05.864 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:00:05.865 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/ae8a8e28-9c94-4613-9bd6-01b7bcd78699 --client_fs=cephfs) 2024-09-16T00:00:05.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-09-16T00:00:05.933 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:00:05.991 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-09-16T00:00:05.991 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-09-16T00:00:05.991 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-09-16T00:00:05.991 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-09-16T00:00:05.991 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-09-16T00:00:05.992 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/18782f8e2f502102cb1a5d248dc8da6b9fdae02e30926ec497ad50b000ff2992/merged 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c5fba984cf18a7bf2d1f64ee2cda5bbeac0ac979101c7b8b46e568420d0a5140/merged 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:05.993 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/95a6553b81a4e718eb37aae9489f0ea17a383d6d2deb7c04a704e1241524cef6/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/372c2f8b36c16df048d603fb7b5a937897ae330bea6e91c54e36dba03eb9a21a/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e4e7cabb51689eb4bb8dd0b237f4f2a7bdfdf79bd8e772372b2fe539193f420c/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b6ffe435bde5f2d3308c25c9d6449f80fec4c43ed1211892c08228d96a35557e/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/eebf208e408df8e642bc0a2912044e2ae5fedb056f473dcf31505a35cca07444/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/6fd3e5e5f8304de5adc7c67fd81c7f1d1f6e9bd7aa26ae2d6c90ff9437d309a2/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/2226907b8c81f952bc1a40c4e5bac91bc987d5dc498feef5397d13eb1e2eaf31/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/208c93059dc4a78a65f56d18419e6c595ca845f6f0378bb95dbc00e32b350c61/merged 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:05.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:05.995 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-09-16T00:00:06.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:05.755+0000 7f6b49a90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:00:06.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:05.821+0000 7f6b49a90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:00:06.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:05.984+0000 7f6b49a90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:00:06.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-09-16T00:00:06.023+0000 7f81dea82480 -1 init, newargv = 0x5565c425fac0 newargc=15 2024-09-16T00:00:06.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[83410]: starting ceph client 2024-09-16T00:00:06.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[83410]: starting fuse 2024-09-16T00:00:06.120 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:05.902+0000 7f5194528040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:00:06.120 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.062+0000 7f5194528040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:00:06.138 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:05.910+0000 7f427e167040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:00:06.139 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.072+0000 7f427e167040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:00:06.139 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.138+0000 7f427e167040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:00:06.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.064+0000 7f6b49a90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:00:06.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.128+0000 7f6b49a90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:00:06.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.295+0000 7f6b49a90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:00:06.401 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.128+0000 7f5194528040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:00:06.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.195+0000 7f5194528040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:00:06.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.278+0000 7f5194528040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:06.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.341+0000 7f5194528040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:00:06.402 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:06.401+0000 7f5194528040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:00:06.411 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.205+0000 7f427e167040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:00:06.411 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.290+0000 7f427e167040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:06.411 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.351+0000 7f427e167040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:00:06.412 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:06.410+0000 7f427e167040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:00:06.644 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.363+0000 7f6b49a90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:00:06.645 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.433+0000 7f6b49a90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:00:06.645 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.519+0000 7f6b49a90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:00:06.645 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.582+0000 7f6b49a90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:00:06.645 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:06.643+0000 7f6b49a90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:00:07.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-09-16T00:00:07.151 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-09-16T00:00:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:00:07.210 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/18782f8e2f502102cb1a5d248dc8da6b9fdae02e30926ec497ad50b000ff2992/merged 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c5fba984cf18a7bf2d1f64ee2cda5bbeac0ac979101c7b8b46e568420d0a5140/merged 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/95a6553b81a4e718eb37aae9489f0ea17a383d6d2deb7c04a704e1241524cef6/merged 2024-09-16T00:00:07.211 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/372c2f8b36c16df048d603fb7b5a937897ae330bea6e91c54e36dba03eb9a21a/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/e4e7cabb51689eb4bb8dd0b237f4f2a7bdfdf79bd8e772372b2fe539193f420c/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/b6ffe435bde5f2d3308c25c9d6449f80fec4c43ed1211892c08228d96a35557e/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/eebf208e408df8e642bc0a2912044e2ae5fedb056f473dcf31505a35cca07444/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/6fd3e5e5f8304de5adc7c67fd81c7f1d1f6e9bd7aa26ae2d6c90ff9437d309a2/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/2226907b8c81f952bc1a40c4e5bac91bc987d5dc498feef5397d13eb1e2eaf31/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/208c93059dc4a78a65f56d18419e6c595ca845f6f0378bb95dbc00e32b350c61/merged 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run.smithi031.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:07.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:07.213 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-09-16T00:00:07.268 INFO:teuthology.orchestra.run.smithi031.stdout:120 2024-09-16T00:00:07.268 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-16T00:00:07.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:00:07.268 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-09-16T00:00:07.268 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-09-16T00:00:07.268 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:00:07.269 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-09-16T00:00:07.270 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-09-16T00:00:07.392 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.83410.asok 2024-09-16T00:00:07.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:00:07.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: Standby manager daemon z restarted 2024-09-16T00:00:07.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: Standby manager daemon z started 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: Standby manager daemon y restarted 2024-09-16T00:00:07.403 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: Standby manager daemon y started 2024-09-16T00:00:07.404 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:00:07.404 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.404 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:00:07.404 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-mon[31203]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.405 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83410.asok 2024-09-16T00:00:07.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:07.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83410.asok status 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y[36423]: 2024-09-16T00:00:07.203+0000 7f427e167040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: Standby manager daemon z restarted 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: Standby manager daemon z started 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: Standby manager daemon y restarted 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: Standby manager daemon y started 2024-09-16T00:00:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:00:07.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:00:07.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:07 smithi081 ceph-mon[35060]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-16T00:00:07.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "83410", 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/volumes/qa/sv_0/ae8a8e28-9c94-4613-9bd6-01b7bcd78699" 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 15228, 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 15228 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:00:07.567 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 908407183 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 908407183 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.15228 192.168.0.1:0/908407183", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/908407183", 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 5, 2024-09-16T00:00:07.568 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 27, 2024-09-16T00:00:07.569 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 80, 2024-09-16T00:00:07.569 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:00:07.569 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-09-16T00:00:07.569 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-09-16T00:00:07.569 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:00:07.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:07.578 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-09-16T00:00:07.578 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:07.596 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-09-16T00:00:07.596 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:07.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:00:07.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:07.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z[36265]: 2024-09-16T00:00:07.190+0000 7f5194528040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: Standby manager daemon z restarted 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: Standby manager daemon z started 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:00:07.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.102:0/3699683255' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: Standby manager daemon y restarted 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: Standby manager daemon y started 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:00:07.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:07 smithi102 ceph-mon[34458]: from='mgr.? 172.21.15.81:0/1849580797' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:00:07.668 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.491+0000 7f6b49a90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:00:07.676 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:00:07.684 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:00:07.684 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:00:07.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-16T00:00:07.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.863+0000 7fc4427fc640 1 -- 172.21.15.31:0/4101772477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438031880 con 0x7fc444101c30 2024-09-16T00:00:07.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 -- 172.21.15.31:0/4101772477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc444101c30 msgr2=0x7fc4441020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:07.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 --2- 172.21.15.31:0/4101772477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc444101c30 0x7fc4441020b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc438009900 tx=0x7fc43802a410 comp rx=0 tx=0).stop 2024-09-16T00:00:07.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 -- 172.21.15.31:0/4101772477 shutdown_connections 2024-09-16T00:00:07.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 --2- 172.21.15.31:0/4101772477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4441025f0 0x7fc444108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:07.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 --2- 172.21.15.31:0/4101772477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc444101c30 0x7fc4441020b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:07.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 --2- 172.21.15.31:0/4101772477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:07.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 -- 172.21.15.31:0/4101772477 >> 172.21.15.31:0/4101772477 conn(0x7fc4440fc1e0 msgr2=0x7fc4440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:07.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 -- 172.21.15.31:0/4101772477 shutdown_connections 2024-09-16T00:00:07.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.864+0000 7fc449e77640 1 -- 172.21.15.31:0/4101772477 wait complete. 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.865+0000 7fc449e77640 1 Processor -- start 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.865+0000 7fc449e77640 1 -- start start 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc444101c30 0x7fc444196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 0x7fc44419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:07.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b460 con 0x7fc4441025f0 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b2e0 con 0x7fc444100a40 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc449e77640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b5e0 con 0x7fc444101c30 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:39214/0 (socket says 172.21.15.31:39214) 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc4437fe640 1 -- 172.21.15.31:0/2215952369 learned_addr learned my addr 172.21.15.31:0/2215952369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc442ffd640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc444101c30 0x7fc444196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 0x7fc44419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 -- 172.21.15.31:0/2215952369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc444101c30 msgr2=0x7fc444196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc444101c30 0x7fc444196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 -- 172.21.15.31:0/2215952369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 msgr2=0x7fc444195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:07.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.866+0000 7fc443fff640 1 -- 172.21.15.31:0/2215952369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44419d700 con 0x7fc4441025f0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc443fff640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 0x7fc44419d090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009fd0 tx=0x7fc43c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00ce20 con 0x7fc4441025f0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44419d9f0 con 0x7fc4441025f0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc43c010430 con 0x7fc4441025f0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44419df00 con 0x7fc4441025f0 2024-09-16T00:00:07.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.867+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00f620 con 0x7fc4441025f0 2024-09-16T00:00:07.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.868+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44419e270 con 0x7fc4441025f0 2024-09-16T00:00:07.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.871+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fc43c0105e0 con 0x7fc4441025f0 2024-09-16T00:00:07.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.872+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7fc43c0cf680 con 0x7fc4441025f0 2024-09-16T00:00:07.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:07.878+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc43c09c740 con 0x7fc4441025f0 2024-09-16T00:00:08.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab42a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab42a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab42a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab42a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab42a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab0861640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab0861640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab0861640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab0861640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:07.790+0000 7f6ab0861640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:00:08.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.099+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fc444107820 con 0x7fc4441025f0 2024-09-16T00:00:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.497+0000 7fc440ff9640 1 -- 172.21.15.31:0/2215952369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fc43c09c040 con 0x7fc4441025f0 2024-09-16T00:00:08.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 msgr2=0x7fc44419d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 0x7fc44419d090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009fd0 tx=0x7fc43c00ef10 comp rx=0 tx=0).stop 2024-09-16T00:00:08.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 shutdown_connections 2024-09-16T00:00:08.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4441025f0 0x7fc44419d090 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc444101c30 0x7fc444196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 --2- 172.21.15.31:0/2215952369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc444100a40 0x7fc444195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.500+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 >> 172.21.15.31:0/2215952369 conn(0x7fc4440fc1e0 msgr2=0x7fc4440fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.501+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 shutdown_connections 2024-09-16T00:00:08.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.501+0000 7fc449e77640 1 -- 172.21.15.31:0/2215952369 wait complete. 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: Active manager daemon x restarted 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: Activating manager daemon x 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: mgrmap e21: x(active, starting, since 0.0128653s), standbys: z, y 2024-09-16T00:00:08.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:00:08.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:00:08.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: Manager daemon x is now available 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:00:08.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:08 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:00:08.517 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T00:00:08.529 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:00:08.545 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:00:08.545 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:00:08.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: Active manager daemon x restarted 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: Activating manager daemon x 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: mgrmap e21: x(active, starting, since 0.0128653s), standbys: z, y 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:00:08.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:00:08.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: Manager daemon x is now available 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:00:08.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:08 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: Active manager daemon x restarted 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: Activating manager daemon x 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: mgrmap e21: x(active, starting, since 0.0128653s), standbys: z, y 2024-09-16T00:00:08.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:00:08.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:00:08.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: Manager daemon x is now available 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:00:08.624 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:08 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-16T00:00:08.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 -- 172.21.15.31:0/1538073470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad98101c30 msgr2=0x7fad981020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/1538073470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad98101c30 0x7fad981020b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009900 tx=0x7fad8c02a410 comp rx=0 tx=0).stop 2024-09-16T00:00:08.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 -- 172.21.15.31:0/1538073470 shutdown_connections 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/1538073470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad981025f0 0x7fad98108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/1538073470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad98101c30 0x7fad981020b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/1538073470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98100a40 0x7fad98100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 -- 172.21.15.31:0/1538073470 >> 172.21.15.31:0/1538073470 conn(0x7fad980fc1e0 msgr2=0x7fad980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 -- 172.21.15.31:0/1538073470 shutdown_connections 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.653+0000 7fad9e0ba640 1 -- 172.21.15.31:0/1538073470 wait complete. 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 Processor -- start 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 -- start start 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad98100a40 0x7fad9806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98101c30 0x7fad9819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad9810c5c0 con 0x7fad981025f0 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad9810c440 con 0x7fad98101c30 2024-09-16T00:00:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad9e0ba640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad9810c740 con 0x7fad98100a40 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54024/0 (socket says 172.21.15.31:54024) 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad97fff640 1 -- 172.21.15.31:0/2798019910 learned_addr learned my addr 172.21.15.31:0/2798019910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.654+0000 7fad977fe640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad98100a40 0x7fad9806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad96ffd640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98101c30 0x7fad9819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 -- 172.21.15.31:0/2798019910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad98100a40 msgr2=0x7fad9806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad98100a40 0x7fad9806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 -- 172.21.15.31:0/2798019910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98101c30 msgr2=0x7fad9819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98101c30 0x7fad9819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 -- 172.21.15.31:0/2798019910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad981a1b70 con 0x7fad981025f0 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad97fff640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad9000eb10 tx=0x7fad9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90004b20 con 0x7fad981025f0 2024-09-16T00:00:08.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad90004cc0 con 0x7fad981025f0 2024-09-16T00:00:08.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad981a1e00 con 0x7fad981025f0 2024-09-16T00:00:08.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90010690 con 0x7fad981025f0 2024-09-16T00:00:08.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.655+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad981a2310 con 0x7fad981025f0 2024-09-16T00:00:08.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.656+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad98107db0 con 0x7fad981025f0 2024-09-16T00:00:08.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.657+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fad90005020 con 0x7fad981025f0 2024-09-16T00:00:08.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.658+0000 7fad94ff9640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 0x7fad800a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.658+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7fad900cfa60 con 0x7fad981025f0 2024-09-16T00:00:08.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.658+0000 7fad977fe640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 0x7fad800a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.658+0000 7fad977fe640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 0x7fad800a5670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad98101a90 tx=0x7fad8800b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:08.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.662+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad9009ca30 con 0x7fad981025f0 2024-09-16T00:00:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.841+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fad98062f00 con 0x7fad981025f0 2024-09-16T00:00:08.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.845+0000 7fad94ff9640 1 -- 172.21.15.31:0/2798019910 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v26)=0 v26) ==== 129+0+0 (secure 0 0 0) 0x7fad9009c330 con 0x7fad981025f0 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 msgr2=0x7fad800a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 0x7fad800a5670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad98101a90 tx=0x7fad8800b390 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 msgr2=0x7fad981a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fad9000eb10 tx=0x7fad9000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 shutdown_connections 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fad800a31b0 0x7fad800a5670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad981025f0 0x7fad981a1380 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad98101c30 0x7fad9819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 --2- 172.21.15.31:0/2798019910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad98100a40 0x7fad9806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 >> 172.21.15.31:0/2798019910 conn(0x7fad980fc1e0 msgr2=0x7fad98104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:08.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 shutdown_connections 2024-09-16T00:00:08.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.848+0000 7fad9e0ba640 1 -- 172.21.15.31:0/2798019910 wait complete. 2024-09-16T00:00:08.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-16T00:00:08.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 -- 172.21.15.31:0/1225831111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 msgr2=0x7f9e581072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/1225831111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581072f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009900 tx=0x7f9e4002a410 comp rx=0 tx=0).stop 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 -- 172.21.15.31:0/1225831111 shutdown_connections 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/1225831111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e58109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/1225831111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581072f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/1225831111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e581049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.968+0000 7f9e5cf86640 1 -- 172.21.15.31:0/1225831111 >> 172.21.15.31:0/1225831111 conn(0x7f9e580fc1b0 msgr2=0x7f9e580fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- 172.21.15.31:0/1225831111 shutdown_connections 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- 172.21.15.31:0/1225831111 wait complete. 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 Processor -- start 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- start start 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e58197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e5819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d6b0 con 0x7f9e58104f00 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d530 con 0x7f9e58107830 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.969+0000 7f9e5cf86640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810d830 con 0x7f9e581025d0 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54042/0 (socket says 172.21.15.31:54042) 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 -- 172.21.15.31:0/3891217833 learned_addr learned my addr 172.21.15.31:0/3891217833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e5659c640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e58197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e56d9d640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e5819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 -- 172.21.15.31:0/3891217833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 msgr2=0x7f9e58197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e58197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 -- 172.21.15.31:0/3891217833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 msgr2=0x7f9e5819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e5819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 -- 172.21.15.31:0/3891217833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5819f980 con 0x7f9e58104f00 2024-09-16T00:00:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e56d9d640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e5819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e5659c640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e58197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4ffff640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e400314e0 tx=0x7f9e40031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4003f030 con 0x7f9e58104f00 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e400044b0 con 0x7f9e58104f00 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5819fc10 con 0x7f9e58104f00 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40035070 con 0x7f9e58104f00 2024-09-16T00:00:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.970+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a0150 con 0x7f9e58104f00 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.972+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e580fdca0 con 0x7f9e58104f00 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.973+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f9e40042030 con 0x7f9e58104f00 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.974+0000 7f9e4f7fe640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 0x7f9e440a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.974+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7f9e400efde0 con 0x7f9e58104f00 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.974+0000 7f9e5659c640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 0x7f9e440a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.974+0000 7f9e5659c640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 0x7f9e440a5640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009fc0 tx=0x7f9e48005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:08.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:08.977+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e400bcdf0 con 0x7f9e58104f00 2024-09-16T00:00:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.157+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9e5802d7c0 con 0x7f9e58104f00 2024-09-16T00:00:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.161+0000 7f9e4f7fe640 1 -- 172.21.15.31:0/3891217833 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f9e400bc6f0 con 0x7f9e58104f00 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 msgr2=0x7f9e440a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 0x7f9e440a5640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009fc0 tx=0x7f9e48005f90 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 msgr2=0x7f9e581982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e400314e0 tx=0x7f9e40031a30 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 shutdown_connections 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9e440a3180 0x7f9e440a5640 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e58107830 0x7f9e5819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e58104f00 0x7f9e581982a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 --2- 172.21.15.31:0/3891217833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e581025d0 0x7f9e58197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 >> 172.21.15.31:0/3891217833 conn(0x7f9e580fc1b0 msgr2=0x7f9e58102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 shutdown_connections 2024-09-16T00:00:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.164+0000 7f9e5cf86640 1 -- 172.21.15.31:0/3891217833 wait complete. 2024-09-16T00:00:09.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 -- 172.21.15.31:0/1263876740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 msgr2=0x7fa64c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 --2- 172.21.15.31:0/1263876740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c109c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa644009ca0 tx=0x7fa64402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 -- 172.21.15.31:0/1263876740 shutdown_connections 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 --2- 172.21.15.31:0/1263876740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c109c90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 --2- 172.21.15.31:0/1263876740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c104f50 0x7fa64c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 --2- 172.21.15.31:0/1263876740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c102620 0x7fa64c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 -- 172.21.15.31:0/1263876740 >> 172.21.15.31:0/1263876740 conn(0x7fa64c0fc1e0 msgr2=0x7fa64c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 -- 172.21.15.31:0/1263876740 shutdown_connections 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.280+0000 7fa65278d640 1 -- 172.21.15.31:0/1263876740 wait complete. 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 Processor -- start 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 -- start start 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c104f50 0x7fa64c1005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10d6b0 con 0x7fa64c107880 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10d530 con 0x7fa64c104f50 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa65278d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10d830 con 0x7fa64c102620 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa650d2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:43892/0 (socket says 172.21.15.31:43892) 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa64b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c104f50 0x7fa64c1005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa64bfff640 1 -- 172.21.15.31:0/868428079 learned_addr learned my addr 172.21.15.31:0/868428079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.281+0000 7fa650d2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54060/0 (socket says 172.21.15.31:54060) 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 -- 172.21.15.31:0/868428079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c104f50 msgr2=0x7fa64c1005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c104f50 0x7fa64c1005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 -- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 msgr2=0x7fa64c100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 -- 172.21.15.31:0/868428079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c1013a0 con 0x7fa64c102620 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa650d2a640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa64bfff640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9f0 tx=0x7fa63c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00cd80 con 0x7fa64c102620 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa63c0108e0 con 0x7fa64c102620 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c1a1e90 con 0x7fa64c102620 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c018920 con 0x7fa64c102620 2024-09-16T00:00:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.282+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a2290 con 0x7fa64c102620 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.283+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c0fdcd0 con 0x7fa64c102620 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.284+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fa63c010a90 con 0x7fa64c102620 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.285+0000 7fa6497fa640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 0x7fa6340a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.285+0000 7fa64b7fe640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 0x7fa6340a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.285+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7fa63c0cfa60 con 0x7fa64c102620 2024-09-16T00:00:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.285+0000 7fa64b7fe640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 0x7fa6340a55c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa64c101d00 tx=0x7fa640005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:09.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.288+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa63c09c3e0 con 0x7fa64c102620 2024-09-16T00:00:09.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.482+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa64c062ed0 con 0x7fa64c102620 2024-09-16T00:00:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.489+0000 7fa6497fa640 1 -- 172.21.15.31:0/868428079 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fa63c0a06e0 con 0x7fa64c102620 2024-09-16T00:00:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.491+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 msgr2=0x7fa6340a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.491+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 0x7fa6340a55c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa64c101d00 tx=0x7fa640005f90 comp rx=0 tx=0).stop 2024-09-16T00:00:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.491+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 msgr2=0x7fa64c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.491+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9f0 tx=0x7fa63c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 shutdown_connections 2024-09-16T00:00:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fa6340a3100 0x7fa6340a55c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c107880 0x7fa64c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa64c104f50 0x7fa64c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 --2- 172.21.15.31:0/868428079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64c102620 0x7fa64c101f90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 >> 172.21.15.31:0/868428079 conn(0x7fa64c0fc1e0 msgr2=0x7fa64c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 shutdown_connections 2024-09-16T00:00:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.492+0000 7fa65278d640 1 -- 172.21.15.31:0/868428079 wait complete. 2024-09-16T00:00:09.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:00:09.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:00:09.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: mgrmap e22: x(active, since 1.02564s), standbys: z, y 2024-09-16T00:00:09.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2798019910' entity='client.admin' 2024-09-16T00:00:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3891217833' entity='client.admin' 2024-09-16T00:00:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-mon[31203]: from='client.? ' entity='client.admin' 2024-09-16T00:00:09.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 -- 172.21.15.31:0/3760750746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 msgr2=0x7faae81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 --2- 172.21.15.31:0/3760750746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae81088c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faadc009930 tx=0x7faadc02a420 comp rx=0 tx=0).stop 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 -- 172.21.15.31:0/3760750746 shutdown_connections 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 --2- 172.21.15.31:0/3760750746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 --2- 172.21.15.31:0/3760750746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 --2- 172.21.15.31:0/3760750746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae81088c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 -- 172.21.15.31:0/3760750746 >> 172.21.15.31:0/3760750746 conn(0x7faae8077490 msgr2=0x7faae80798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 -- 172.21.15.31:0/3760750746 shutdown_connections 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.642+0000 7faaef749640 1 -- 172.21.15.31:0/3760750746 wait complete. 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 Processor -- start 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 -- start start 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faae8113cd0 con 0x7faae81084c0 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faae8113b50 con 0x7faae8109380 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaef749640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faae8113e50 con 0x7faae8109d40 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaed4e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaed4e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54078/0 (socket says 172.21.15.31:54078) 2024-09-16T00:00:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaed4e5640 1 -- 172.21.15.31:0/4174435040 learned_addr learned my addr 172.21.15.31:0/4174435040 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaedce6640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 -- 172.21.15.31:0/4174435040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 msgr2=0x7faae81a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 -- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 msgr2=0x7faae819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaecce4640 1 -- 172.21.15.31:0/4174435040 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81a5fd0 con 0x7faae8109380 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaed4e5640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.643+0000 7faaedce6640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faaecce4640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae819e8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faae000eb00 tx=0x7faae000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0010430 con 0x7faae8109380 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae0010a80 con 0x7faae8109380 2024-09-16T00:00:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae81a62c0 con 0x7faae8109380 2024-09-16T00:00:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0017ad0 con 0x7faae8109380 2024-09-16T00:00:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.644+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae81a6860 con 0x7faae8109380 2024-09-16T00:00:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.645+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae810f620 con 0x7faae8109380 2024-09-16T00:00:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.647+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7faae0017c70 con 0x7faae8109380 2024-09-16T00:00:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.648+0000 7faada7fc640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 0x7faad40a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.648+0000 7faaed4e5640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 0x7faad40a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.648+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7faae00cfe20 con 0x7faae8109380 2024-09-16T00:00:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.648+0000 7faaed4e5640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 0x7faad40a5640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faadc005ec0 tx=0x7faadc005e50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:09.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.652+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faae009ce30 con 0x7faae8109380 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:09.514+0000 7f6a9d2bb640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:00:09] ENGINE Error in HTTPServer.serve 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Traceback (most recent call last): 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._connections.run(self.expiration_interval) 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._run(expiration_interval) 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:00:09.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: s = self.context.wrap_socket( 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: return self.sslsocket_class._create( 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.do_handshake() 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self._sslobj.do_handshake() 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:00:09.756 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:00:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: mgrmap e22: x(active, since 1.02564s), standbys: z, y 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2798019910' entity='client.admin' 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3891217833' entity='client.admin' 2024-09-16T00:00:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:09 smithi081 ceph-mon[35060]: from='client.? ' entity='client.admin' 2024-09-16T00:00:09.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2215952369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-16T00:00:09.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:00:09.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: mgrmap e22: x(active, since 1.02564s), standbys: z, y 2024-09-16T00:00:09.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:09.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2798019910' entity='client.admin' 2024-09-16T00:00:09.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3891217833' entity='client.admin' 2024-09-16T00:00:09.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:09 smithi102 ceph-mon[34458]: from='client.? ' entity='client.admin' 2024-09-16T00:00:09.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.886+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7faae8062630 con 0x7faad40a3180 2024-09-16T00:00:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.950+0000 7faada7fc640 1 -- 172.21.15.31:0/4174435040 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faae8062630 con 0x7faad40a3180 2024-09-16T00:00:09.951 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2024-09-16T00:00:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 msgr2=0x7faad40a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 0x7faad40a5640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faadc005ec0 tx=0x7faadc005e50 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 msgr2=0x7faae819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae819e8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faae000eb00 tx=0x7faae000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 shutdown_connections 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7faad40a3180 0x7faad40a5640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae8109d40 0x7faae81a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae8109380 0x7faae819e8f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 --2- 172.21.15.31:0/4174435040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faae81084c0 0x7faae819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.954+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 >> 172.21.15.31:0/4174435040 conn(0x7faae8077490 msgr2=0x7faae810c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.955+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 shutdown_connections 2024-09-16T00:00:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:09.955+0000 7faaef749640 1 -- 172.21.15.31:0/4174435040 wait complete. 2024-09-16T00:00:09.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-16T00:00:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 -- 172.21.15.31:0/1781062485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b8104f70 msgr2=0x7fd4b8107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 --2- 172.21.15.31:0/1781062485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b8104f70 0x7fd4b8107360 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009900 tx=0x7fd4ac02a410 comp rx=0 tx=0).stop 2024-09-16T00:00:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 -- 172.21.15.31:0/1781062485 shutdown_connections 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 --2- 172.21.15.31:0/1781062485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b81078a0 0x7fd4b8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 --2- 172.21.15.31:0/1781062485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b8104f70 0x7fd4b8107360 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 --2- 172.21.15.31:0/1781062485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 -- 172.21.15.31:0/1781062485 >> 172.21.15.31:0/1781062485 conn(0x7fd4b80fc1e0 msgr2=0x7fd4b80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 -- 172.21.15.31:0/1781062485 shutdown_connections 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.082+0000 7fd4be850640 1 -- 172.21.15.31:0/1781062485 wait complete. 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.083+0000 7fd4be850640 1 Processor -- start 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.083+0000 7fd4be850640 1 -- start start 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.083+0000 7fd4be850640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.083+0000 7fd4be850640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.083+0000 7fd4be850640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b81078a0 0x7fd4b819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4be850640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810d680 con 0x7fd4b81078a0 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4be850640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810d500 con 0x7fd4b8102640 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4be850640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b810d800 con 0x7fd4b8104f70 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4bcded640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b81078a0 0x7fd4b819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.31:43954/0 (socket says 172.21.15.31:43954) 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 -- 172.21.15.31:0/2731294933 learned_addr learned my addr 172.21.15.31:0/2731294933 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 -- 172.21.15.31:0/2731294933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 msgr2=0x7fd4b8197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 -- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b81078a0 msgr2=0x7fd4b819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b81078a0 0x7fd4b819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 -- 172.21.15.31:0/2731294933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b819f9c0 con 0x7fd4b8104f70 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b7fff640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b77fe640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0314e0 tx=0x7fd4ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.084+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac03f030 con 0x7fd4b8104f70 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.085+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b819fc50 con 0x7fd4b8104f70 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.085+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4ac003aa0 con 0x7fd4b8104f70 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.085+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac035070 con 0x7fd4b8104f70 2024-09-16T00:00:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.085+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a0190 con 0x7fd4b8104f70 2024-09-16T00:00:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.086+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b80fdcd0 con 0x7fd4b8104f70 2024-09-16T00:00:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.087+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fd4ac042030 con 0x7fd4b8104f70 2024-09-16T00:00:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.087+0000 7fd4b57fa640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 0x7fd4a00a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.088+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7fd4ac0efc90 con 0x7fd4b8104f70 2024-09-16T00:00:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.088+0000 7fd4b7fff640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 0x7fd4a00a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.088+0000 7fd4b7fff640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 0x7fd4a00a55c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8199280 tx=0x7fd4a8005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.092+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4ac0bc650 con 0x7fd4b8104f70 2024-09-16T00:00:10.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:10.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:10.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84431]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:10.254 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84431]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.332+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd4b8062790 con 0x7fd4a00a3100 2024-09-16T00:00:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.398+0000 7fd4b57fa640 1 -- 172.21.15.31:0/2731294933 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd4b8062790 con 0x7fd4a00a3100 2024-09-16T00:00:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 msgr2=0x7fd4a00a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 0x7fd4a00a55c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8199280 tx=0x7fd4a8005f90 comp rx=0 tx=0).stop 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 msgr2=0x7fd4b81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0314e0 tx=0x7fd4ac004270 comp rx=0 tx=0).stop 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 shutdown_connections 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd4a00a3100 0x7fd4a00a55c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4b81078a0 0x7fd4b819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4b8104f70 0x7fd4b81982f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 --2- 172.21.15.31:0/2731294933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4b8102640 0x7fd4b8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 >> 172.21.15.31:0/2731294933 conn(0x7fd4b80fc1e0 msgr2=0x7fd4b8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 shutdown_connections 2024-09-16T00:00:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.400+0000 7fd4be850640 1 -- 172.21.15.31:0/2731294933 wait complete. 2024-09-16T00:00:10.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-16T00:00:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 -- 172.21.15.31:0/3163225839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274102d50 msgr2=0x7f72741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 --2- 172.21.15.31:0/3163225839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274102d50 0x7f72741031d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402a410 comp rx=0 tx=0).stop 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 -- 172.21.15.31:0/3163225839 shutdown_connections 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 --2- 172.21.15.31:0/3163225839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 --2- 172.21.15.31:0/3163225839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274102d50 0x7f72741031d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 --2- 172.21.15.31:0/3163225839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274101b60 0x7f7274101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 -- 172.21.15.31:0/3163225839 >> 172.21.15.31:0/3163225839 conn(0x7f72740fd2e0 msgr2=0x7f72740ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 -- 172.21.15.31:0/3163225839 shutdown_connections 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.528+0000 7f727bb59640 1 -- 172.21.15.31:0/3163225839 wait complete. 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 Processor -- start 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 -- start start 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f727410d6c0 con 0x7f7274101b60 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f727410d540 con 0x7f7274102d50 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f727bb59640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f727410d840 con 0x7f7274103710 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f72798f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f72790f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f72798f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54124/0 (socket says 172.21.15.31:54124) 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f72798f5640 1 -- 172.21.15.31:0/1497217186 learned_addr learned my addr 172.21.15.31:0/1497217186 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.529+0000 7f72790f4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:39346/0 (socket says 172.21.15.31:39346) 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f727a0f6640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 -- 172.21.15.31:0/1497217186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 msgr2=0x7f7274074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 -- 172.21.15.31:0/1497217186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 msgr2=0x7f72740745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 -- 172.21.15.31:0/1497217186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72740753b0 con 0x7f7274101b60 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f727a0f6640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72790f4640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f72798f5640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9c0 tx=0x7f727000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000cd80 con 0x7f7274101b60 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72700035f0 con 0x7f7274101b60 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741a1e10 con 0x7f7274101b60 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270010690 con 0x7f7274101b60 2024-09-16T00:00:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.530+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741a2270 con 0x7f7274101b60 2024-09-16T00:00:10.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.532+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7274108e80 con 0x7f7274101b60 2024-09-16T00:00:10.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.534+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 138475+0+0 (secure 0 0 0) 0x7f72700037a0 con 0x7f7274101b60 2024-09-16T00:00:10.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.535+0000 7f726affd640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 0x7f72600a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.535+0000 7f72790f4640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 0x7f72600a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.535+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7f72700d0140 con 0x7f7274101b60 2024-09-16T00:00:10.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.536+0000 7f72790f4640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 0x7f72600a59a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72640314e0 tx=0x7f7264005a20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.540+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f727009cc40 con 0x7f7274101b60 2024-09-16T00:00:10.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:10.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:10.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84592]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:10.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:00:10 smithi031 sudo[84592]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: [16/Sep/2024:00:00:09] ENGINE Bus STARTING 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: [16/Sep/2024:00:00:09] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: [16/Sep/2024:00:00:09] ENGINE Error in HTTPServer.serve 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:00:10.755 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:00:10.756 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: [16/Sep/2024:00:00:09] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: [16/Sep/2024:00:00:09] ENGINE Bus STARTED 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:00:10.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:10.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:10 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.761+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7274062630 con 0x7f72600a34e0 2024-09-16T00:00:10.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.768+0000 7f726affd640 1 -- 172.21.15.31:0/1497217186 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7274062630 con 0x7f72600a34e0 2024-09-16T00:00:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-16T00:00:00", "created": "2024-09-16T00:00:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-16T00:00:10.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.770+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 msgr2=0x7f72600a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.770+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 0x7f72600a59a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72640314e0 tx=0x7f7264005a20 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.770+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 msgr2=0x7f7274075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.770+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9c0 tx=0x7f727000ee80 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 shutdown_connections 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f72600a34e0 0x7f72600a59a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7274103710 0x7f7274074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7274102d50 0x7f72740745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 --2- 172.21.15.31:0/1497217186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7274101b60 0x7f7274075fa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 >> 172.21.15.31:0/1497217186 conn(0x7f72740fd2e0 msgr2=0x7f7274106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 shutdown_connections 2024-09-16T00:00:10.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.771+0000 7f727bb59640 1 -- 172.21.15.31:0/1497217186 wait complete. 2024-09-16T00:00:10.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-16T00:00:10.885 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:10 smithi081 sudo[66877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:10.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: [16/Sep/2024:00:00:09] ENGINE Bus STARTING 2024-09-16T00:00:10.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: [16/Sep/2024:00:00:09] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T00:00:10.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:10.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: [16/Sep/2024:00:00:09] ENGINE Error in HTTPServer.serve 2024-09-16T00:00:10.886 INFO:journalctl@ceph.mon.b.smithi081.stdout: Traceback (most recent call last): 2024-09-16T00:00:10.887 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:00:10.887 INFO:journalctl@ceph.mon.b.smithi081.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:00:10.887 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:00:10.887 INFO:journalctl@ceph.mon.b.smithi081.stdout: self._run(expiration_interval) 2024-09-16T00:00:10.887 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: s = self.context.wrap_socket( 2024-09-16T00:00:10.888 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:00:10.889 INFO:journalctl@ceph.mon.b.smithi081.stdout: return self.sslsocket_class._create( 2024-09-16T00:00:10.889 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:00:10.889 INFO:journalctl@ceph.mon.b.smithi081.stdout: self.do_handshake() 2024-09-16T00:00:10.889 INFO:journalctl@ceph.mon.b.smithi081.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:00:10.889 INFO:journalctl@ceph.mon.b.smithi081.stdout: self._sslobj.do_handshake() 2024-09-16T00:00:10.890 INFO:journalctl@ceph.mon.b.smithi081.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:00:10.890 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: [16/Sep/2024:00:00:09] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T00:00:10.890 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: [16/Sep/2024:00:00:09] ENGINE Bus STARTED 2024-09-16T00:00:10.890 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.890 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.891 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.891 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.891 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.892 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.892 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:10.892 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.893 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.893 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.893 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.893 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.893 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.896 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.896 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:10.896 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:10.896 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:10 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:10.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 -- 172.21.15.31:0/385623524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8510101c30 msgr2=0x7f85101020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 --2- 172.21.15.31:0/385623524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8510101c30 0x7f85101020b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8504009900 tx=0x7f850402a410 comp rx=0 tx=0).stop 2024-09-16T00:00:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 -- 172.21.15.31:0/385623524 shutdown_connections 2024-09-16T00:00:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 --2- 172.21.15.31:0/385623524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85101025f0 0x7f8510108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 --2- 172.21.15.31:0/385623524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8510101c30 0x7f85101020b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 --2- 172.21.15.31:0/385623524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 -- 172.21.15.31:0/385623524 >> 172.21.15.31:0/385623524 conn(0x7f85100fc1e0 msgr2=0x7f85100fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 -- 172.21.15.31:0/385623524 shutdown_connections 2024-09-16T00:00:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.891+0000 7f8515ab7640 1 -- 172.21.15.31:0/385623524 wait complete. 2024-09-16T00:00:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 Processor -- start 2024-09-16T00:00:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 -- start start 2024-09-16T00:00:10.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8510101c30 0x7f8510197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 0x7f851019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f851010c550 con 0x7f85101025f0 2024-09-16T00:00:10.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f851010c3d0 con 0x7f8510100a40 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.892+0000 7f8515ab7640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f851010c6d0 con 0x7f8510101c30 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 0x7f851019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.31:39368/0 (socket says 172.21.15.31:39368) 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 -- 172.21.15.31:0/2786480792 learned_addr learned my addr 172.21.15.31:0/2786480792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850effd640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8510101c30 0x7f8510197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 -- 172.21.15.31:0/2786480792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8510101c30 msgr2=0x7f8510197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8510101c30 0x7f8510197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 -- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 msgr2=0x7f851019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 0x7f851019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 -- 172.21.15.31:0/2786480792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851019e830 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850ffff640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 0x7f851019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850f7fe640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f850000e9c0 tx=0x7f850000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850000cd80 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851019eb20 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85000035f0 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.893+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500010690 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.894+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851019f070 con 0x7f8510100a40 2024-09-16T00:00:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.895+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8510107e30 con 0x7f8510100a40 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.896+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 23) ==== 138475+0+0 (secure 0 0 0) 0x7f85000037a0 con 0x7f8510100a40 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.897+0000 7f850cff9640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 0x7f84f80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.897+0000 7f850effd640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 0x7f84f80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.897+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8237+0+0 (secure 0 0 0) 0x7f85000cfff0 con 0x7f8510100a40 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.898+0000 7f850effd640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 0x7f84f80a5920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f8504000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:00:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:10.902+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f850009caf0 con 0x7f8510100a40 2024-09-16T00:00:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: [16/Sep/2024:00:00:09] ENGINE Bus STARTING 2024-09-16T00:00:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: [16/Sep/2024:00:00:09] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T00:00:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: [16/Sep/2024:00:00:09] ENGINE Error in HTTPServer.serve 2024-09-16T00:00:11.121 INFO:journalctl@ceph.mon.c.smithi102.stdout: Traceback (most recent call last): 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._run(expiration_interval) 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: s = self.context.wrap_socket( 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:00:11.122 INFO:journalctl@ceph.mon.c.smithi102.stdout: return self.sslsocket_class._create( 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout: self.do_handshake() 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._sslobj.do_handshake() 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: [16/Sep/2024:00:00:09] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: [16/Sep/2024:00:00:09] ENGINE Bus STARTED 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.123 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:00:11.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:11.126 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:10 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.121+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8510062630 con 0x7f84f80a3460 2024-09-16T00:00:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.127+0000 7f850cff9640 1 -- 172.21.15.31:0/2786480792 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8510062630 con 0x7f84f80a3460 2024-09-16T00:00:11.128 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-16T00:00:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 msgr2=0x7f84f80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 0x7f84f80a5920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8504006010 tx=0x7f8504000bc0 comp rx=0 tx=0).stop 2024-09-16T00:00:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 msgr2=0x7f8510196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f850000e9c0 tx=0x7f850000ee80 comp rx=0 tx=0).stop 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 shutdown_connections 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f84f80a3460 0x7f84f80a5920 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85101025f0 0x7f851019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8510101c30 0x7f8510197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 --2- 172.21.15.31:0/2786480792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8510100a40 0x7f8510196c20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 >> 172.21.15.31:0/2786480792 conn(0x7f85100fc1e0 msgr2=0x7f8510105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 shutdown_connections 2024-09-16T00:00:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:00:11.130+0000 7f8515ab7640 1 -- 172.21.15.31:0/2786480792 wait complete. 2024-09-16T00:00:11.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-16T00:00:11.157 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:10 smithi081 sudo[66877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:11.157 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:10 smithi081 sudo[66877]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:11.157 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:10 smithi081 sudo[66877]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:11.173 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-16T00:00:11.183 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T00:00:11.230 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:00:11.230 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:00:11.230 INFO:tasks.workunit:timeout=3h 2024-09-16T00:00:11.230 INFO:tasks.workunit:cleanup=True 2024-09-16T00:00:11.230 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:11.248 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-15 23:58:28.594922567 +0000 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-15 23:58:28.594922567 +0000 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 00:00:07.672367940 +0000 2024-09-16T00:00:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-09-16T00:00:11.249 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:00:11.249 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:00:11.323 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:00:11.380 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:00:11.556 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:11 smithi081 sudo[67096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:11.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:11 smithi081 sudo[67096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:11.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:11 smithi081 sudo[67096]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:11.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:00:11 smithi081 sudo[67096]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:11.711 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67518]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:11.711 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67518]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:11.711 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67518]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:11.711 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67518]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:11.711 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:11.712 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:11.713 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:11 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:11 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi081:/etc/ceph/ceph.conf 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:12.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: Updating smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.058 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.059 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:11 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.120 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:12.120 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:12.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67664]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:12.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:00:11 smithi102 sudo[67664]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:12.504 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:12.504 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:12.504 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85587]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:12.505 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85587]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:12.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:12 smithi081 ceph-mon[35060]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-16T00:00:13.004 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:13.004 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:13.004 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85591]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:13.004 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:00:12 smithi031 sudo[85591]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:13.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:12 smithi031 ceph-mon[31203]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-16T00:00:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:00:13.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:12 smithi102 ceph-mon[34458]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-16T00:00:13.145 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:12 smithi081 sudo[67540]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:13.145 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:12 smithi081 sudo[67540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:13.145 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:12 smithi081 sudo[67540]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:13.145 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:12 smithi081 sudo[67540]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:13.557 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:13 smithi081 sudo[67543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:13.557 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:13 smithi081 sudo[67543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:13.557 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:13 smithi081 sudo[67543]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:13.557 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:00:13 smithi081 sudo[67543]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:13.653 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:13.653 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:13.653 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67699]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:13.653 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67699]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:14.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:14.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:14.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67702]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:14.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:00:13 smithi102 sudo[67702]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:14.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:14.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:14.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85605]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:14.505 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85605]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:14.959 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85609]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:14.959 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:14.960 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85609]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:14.960 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:00:14 smithi031 sudo[85609]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:15.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:14 smithi081 ceph-mon[35060]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:15.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:14 smithi081 sudo[67546]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:15.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:14 smithi081 sudo[67546]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:15.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:14 smithi081 sudo[67546]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:15.057 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:14 smithi081 sudo[67546]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:15.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:14 smithi031 ceph-mon[31203]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:15.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:14 smithi102 ceph-mon[34458]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:00:15.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:15 smithi081 sudo[67549]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:15.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:15 smithi081 sudo[67549]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:15.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:15 smithi081 sudo[67549]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:15.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:00:15 smithi081 sudo[67549]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:15.573 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:15.573 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:15.573 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67705]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:15.574 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67705]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:15.870 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:15.870 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:15.870 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67708]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:00:15 smithi102 sudo[67708]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:16.479 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85612]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:16.479 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85612]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:16.479 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85612]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:16.479 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85612]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:16.754 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:16.754 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:16.754 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85615]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:16.755 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:00:16 smithi031 sudo[85615]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:16.997 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:16 smithi081 ceph-mon[35060]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 300 KiB/s wr, 93 op/s 2024-09-16T00:00:16.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:16.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:16.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67552]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:16.997 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67552]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:17.246 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:16 smithi102 ceph-mon[34458]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 300 KiB/s wr, 93 op/s 2024-09-16T00:00:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:16 smithi031 ceph-mon[31203]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 300 KiB/s wr, 93 op/s 2024-09-16T00:00:17.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67555]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:17.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67555]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:17.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67555]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:17.308 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:00:16 smithi081 sudo[67555]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:17.504 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:17.826 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:17.827 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67711]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:17.827 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67711]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:17.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:17.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:17.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67714]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:17.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:00:17 smithi102 sudo[67714]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:18.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:17 smithi031 sudo[85618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:18.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:17 smithi031 sudo[85618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:18.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:17 smithi031 sudo[85618]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:18.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:17 smithi031 sudo[85618]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:18.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 sudo[85621]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:18.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 sudo[85621]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:18.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 sudo[85621]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:18.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 sudo[85621]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:18.910 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:18.910 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:18.910 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67717]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:18.911 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67717]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 233 KiB/s wr, 73 op/s 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:00:19.178 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:19.179 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67558]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:19.179 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67558]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 233 KiB/s wr, 73 op/s 2024-09-16T00:00:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67720]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67720]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 sudo[67720]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 233 KiB/s wr, 73 op/s 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67561]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:00:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67561]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:00:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67561]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:00:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 sudo[67561]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:00:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:19 smithi031 ceph-mon[31203]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:20.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:19 smithi081 ceph-mon[35060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:20.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:00:20.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:19 smithi102 ceph-mon[34458]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:00:21.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:20 smithi031 ceph-mon[31203]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 191 KiB/s wr, 64 op/s 2024-09-16T00:00:21.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:20 smithi081 ceph-mon[35060]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 191 KiB/s wr, 64 op/s 2024-09-16T00:00:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:20 smithi102 ceph-mon[34458]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 191 KiB/s wr, 64 op/s 2024-09-16T00:00:23.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:22 smithi031 ceph-mon[31203]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:23.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:22 smithi081 ceph-mon[35060]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:22 smithi102 ceph-mon[34458]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:25.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:24 smithi031 ceph-mon[31203]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:25.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:24 smithi081 ceph-mon[35060]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:24 smithi102 ceph-mon[34458]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-16T00:00:27.199 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:26 smithi102 ceph-mon[34458]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 197 KiB/s wr, 65 op/s 2024-09-16T00:00:27.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:26 smithi031 ceph-mon[31203]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 197 KiB/s wr, 65 op/s 2024-09-16T00:00:27.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:26 smithi081 ceph-mon[35060]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 197 KiB/s wr, 65 op/s 2024-09-16T00:00:29.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:28 smithi031 ceph-mon[31203]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:29.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:28 smithi081 ceph-mon[35060]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:28 smithi102 ceph-mon[34458]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:31.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:30 smithi031 ceph-mon[31203]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:31.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:30 smithi081 ceph-mon[35060]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:30 smithi102 ceph-mon[34458]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:33.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:32 smithi031 ceph-mon[31203]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:33.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:32 smithi081 ceph-mon[35060]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:32 smithi102 ceph-mon[34458]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:35.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:34 smithi031 ceph-mon[31203]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:35.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:34 smithi081 ceph-mon[35060]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:34 smithi102 ceph-mon[34458]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-09-16T00:00:37.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:36 smithi031 ceph-mon[31203]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:37.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:36 smithi081 ceph-mon[35060]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:36 smithi102 ceph-mon[34458]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-16T00:00:39.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:38 smithi031 ceph-mon[31203]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:39.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:38 smithi081 ceph-mon[35060]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:38 smithi102 ceph-mon[34458]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:41.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:40 smithi031 ceph-mon[31203]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:41.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:40 smithi081 ceph-mon[35060]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:40 smithi102 ceph-mon[34458]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:42 smithi031 ceph-mon[31203]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:43.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:42 smithi081 ceph-mon[35060]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:43.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:42 smithi102 ceph-mon[34458]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:45.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:44 smithi031 ceph-mon[31203]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:45.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:44 smithi081 ceph-mon[35060]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:45.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:44 smithi102 ceph-mon[34458]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:47.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:46 smithi031 ceph-mon[31203]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:47.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:46 smithi081 ceph-mon[35060]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:47.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:46 smithi102 ceph-mon[34458]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:48 smithi031 ceph-mon[31203]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:49.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:48 smithi081 ceph-mon[35060]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:48 smithi102 ceph-mon[34458]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:50 smithi031 ceph-mon[31203]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:51.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:50 smithi081 ceph-mon[35060]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:50 smithi102 ceph-mon[34458]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:52 smithi031 ceph-mon[31203]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:53.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:52 smithi081 ceph-mon[35060]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:53.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:52 smithi102 ceph-mon[34458]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:54 smithi031 ceph-mon[31203]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:55.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:54 smithi081 ceph-mon[35060]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:55.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:54 smithi102 ceph-mon[34458]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:56 smithi031 ceph-mon[31203]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:57.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:56 smithi081 ceph-mon[35060]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:57.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:56 smithi102 ceph-mon[34458]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:00:59.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:00:58 smithi031 ceph-mon[31203]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:59.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:00:58 smithi081 ceph-mon[35060]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:00:59.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:00:58 smithi102 ceph-mon[34458]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:01:00.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:01:00.409+0000 7f6a8d95c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:01:00.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:01:00.409+0000 7f6a8d95c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:01:00.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:01:00.410+0000 7f6a8d95c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:01:00.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:01:00.410+0000 7f6a8d95c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:01:00.755 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:01:00.410+0000 7f6a8d95c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:01:01.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:00 smithi031 ceph-mon[31203]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:01:01.289 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:00 smithi102 ceph-mon[34458]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:01:01.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:00 smithi081 ceph-mon[35060]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:01:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:02 smithi031 ceph-mon[31203]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:01:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:02 smithi031 ceph-mon[31203]: mgrmap e25: x(active, since 54s), standbys: z, y 2024-09-16T00:01:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:02 smithi081 ceph-mon[35060]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:01:03.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:02 smithi081 ceph-mon[35060]: mgrmap e25: x(active, since 54s), standbys: z, y 2024-09-16T00:01:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:02 smithi102 ceph-mon[34458]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:01:03.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:02 smithi102 ceph-mon[34458]: mgrmap e25: x(active, since 54s), standbys: z, y 2024-09-16T00:01:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:04 smithi031 ceph-mon[31203]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-16T00:01:05.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:04 smithi081 ceph-mon[35060]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-16T00:01:05.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:04 smithi102 ceph-mon[34458]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-16T00:01:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:06 smithi031 ceph-mon[31203]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-16T00:01:07.306 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:06 smithi081 ceph-mon[35060]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-16T00:01:07.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:06 smithi102 ceph-mon[34458]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-16T00:01:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T00:01:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:01:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:01:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:01:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-16T00:01:08.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:07 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-16T00:01:08.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:07 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-09-16T00:01:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-16T00:01:08.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:01:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:01:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:01:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-16T00:01:08.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:07 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-16T00:01:09.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:09 smithi081 ceph-mon[35060]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:01:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T00:01:09.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-16T00:01:09.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:09 smithi102 ceph-mon[34458]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:01:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-16T00:01:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-16T00:01:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:01:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:01:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:01:09.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-16T00:01:09.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-16T00:01:09.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:09 smithi031 ceph-mon[31203]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:01:10.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:10 smithi081 ceph-mon[35060]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:01:10.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:10 smithi102 ceph-mon[34458]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:01:10.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:10 smithi031 ceph-mon[31203]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:01:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:11 smithi081 ceph-mon[35060]: pgmap v36: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 36 op/s 2024-09-16T00:01:11.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:11 smithi081 ceph-mon[35060]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:01:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:11 smithi102 ceph-mon[34458]: pgmap v36: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 36 op/s 2024-09-16T00:01:11.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:11 smithi102 ceph-mon[34458]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:01:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:11 smithi031 ceph-mon[31203]: pgmap v36: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 36 op/s 2024-09-16T00:01:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:11 smithi031 ceph-mon[31203]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T00:01:12.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:12 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:01:12.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:12 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:01:12.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:12 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:01:13.041 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 93% (12044/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T00:01:13.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: pgmap v37: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-09-16T00:01:13.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:01:13.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:01:13.406 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:01:13.411 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T00:01:13.412 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:01:13.416 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:01:13.477 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:01:13.479 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:01:13.479 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: pgmap v37: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:01:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: pgmap v37: 97 pgs: 6 peering, 91 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-09-16T00:01:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:01:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:01:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:01:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:01:14.224 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:01:14.267 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:01:14.268 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:01:14.268 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:01:14.308 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:01:14.312 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:01:14.312 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:01:14.369 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-16T00:01:14.371 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-16T00:01:14.371 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-16T00:01:14.436 INFO:tasks.workunit.client.0.smithi031.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-16T00:01:14.508 INFO:tasks.workunit.client.0.smithi031.stdout: Iozone: Performance Test of File I/O 2024-09-16T00:01:14.508 INFO:tasks.workunit.client.0.smithi031.stdout: Version $Revision: 3.494 $ 2024-09-16T00:01:14.508 INFO:tasks.workunit.client.0.smithi031.stdout: Compiled for 64 bit mode. 2024-09-16T00:01:14.508 INFO:tasks.workunit.client.0.smithi031.stdout: Build: linux-AMD64 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: Run began: Mon Sep 16 00:01:14 2024 2024-09-16T00:01:14.509 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Include close in write timing 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Include fsync in write timing 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: File size set to 1048576 kB 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Record Size 16 kB 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Output is in kBytes/sec 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache line size set to 32 bytes. 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: File stride size set to 17 * record size. 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Throughput test with 1 process 2024-09-16T00:01:14.510 INFO:tasks.workunit.client.0.smithi031.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-16T00:01:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:15 smithi031 ceph-mon[31203]: pgmap v38: 97 pgs: 5 peering, 92 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 9 op/s 2024-09-16T00:01:15.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:15 smithi081 ceph-mon[35060]: pgmap v38: 97 pgs: 5 peering, 92 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 9 op/s 2024-09-16T00:01:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:15 smithi102 ceph-mon[34458]: pgmap v38: 97 pgs: 5 peering, 92 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 9 op/s 2024-09-16T00:01:16.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:16 smithi031 ceph-mon[31203]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T00:01:16.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:16 smithi031 ceph-mon[31203]: Cluster is now healthy 2024-09-16T00:01:16.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:16 smithi081 ceph-mon[35060]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T00:01:16.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:16 smithi081 ceph-mon[35060]: Cluster is now healthy 2024-09-16T00:01:16.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:16 smithi102 ceph-mon[34458]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T00:01:16.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:16 smithi102 ceph-mon[34458]: Cluster is now healthy 2024-09-16T00:01:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:17 smithi031 ceph-mon[31203]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T00:01:17.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:17 smithi081 ceph-mon[35060]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T00:01:17.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:17 smithi102 ceph-mon[34458]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-16T00:01:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:19 smithi031 ceph-mon[31203]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 107 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:01:19.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:19 smithi081 ceph-mon[35060]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 107 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:01:19.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:19 smithi102 ceph-mon[34458]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 107 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-16T00:01:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:21 smithi031 ceph-mon[31203]: pgmap v41: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 37 MiB/s wr, 20 op/s; 10 B/s, 0 objects/s recovering 2024-09-16T00:01:21.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:21 smithi081 ceph-mon[35060]: pgmap v41: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 37 MiB/s wr, 20 op/s; 10 B/s, 0 objects/s recovering 2024-09-16T00:01:21.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:21 smithi102 ceph-mon[34458]: pgmap v41: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 37 MiB/s wr, 20 op/s; 10 B/s, 0 objects/s recovering 2024-09-16T00:01:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:23 smithi031 ceph-mon[31203]: pgmap v42: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 33 MiB/s wr, 17 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:23.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:23 smithi081 ceph-mon[35060]: pgmap v42: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 33 MiB/s wr, 17 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:23.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:23 smithi102 ceph-mon[34458]: pgmap v42: 97 pgs: 97 active+clean; 393 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 33 MiB/s wr, 17 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:24.454 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 initial writers = 105457.77 kB/sec 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 initial writers = 105455.39 kB/sec 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 105457.77 kB/sec 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 105457.77 kB/sec 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 105457.77 kB/sec 2024-09-16T00:01:24.455 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:25 smithi031 ceph-mon[31203]: pgmap v43: 97 pgs: 97 active+clean; 833 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 69 MiB/s wr, 30 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:25.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:25 smithi081 ceph-mon[35060]: pgmap v43: 97 pgs: 97 active+clean; 833 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 69 MiB/s wr, 30 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:25 smithi102 ceph-mon[34458]: pgmap v43: 97 pgs: 97 active+clean; 833 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 69 MiB/s wr, 30 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:27 smithi031 ceph-mon[31203]: pgmap v44: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 31 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:27.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:27 smithi081 ceph-mon[35060]: pgmap v44: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 31 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:27.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:27 smithi102 ceph-mon[34458]: pgmap v44: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 31 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T00:01:29.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:29 smithi031 ceph-mon[31203]: pgmap v45: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 MiB/s wr, 29 op/s 2024-09-16T00:01:29.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:29 smithi081 ceph-mon[35060]: pgmap v45: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 MiB/s wr, 29 op/s 2024-09-16T00:01:29.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:29 smithi102 ceph-mon[34458]: pgmap v45: 97 pgs: 97 active+clean; 921 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 MiB/s wr, 29 op/s 2024-09-16T00:01:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:31 smithi031 ceph-mon[31203]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-16T00:01:31.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:31 smithi081 ceph-mon[35060]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-16T00:01:31.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:31 smithi102 ceph-mon[34458]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-16T00:01:33.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:33 smithi031 ceph-mon[31203]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 53 MiB/s wr, 22 op/s 2024-09-16T00:01:33.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:33 smithi081 ceph-mon[35060]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 53 MiB/s wr, 22 op/s 2024-09-16T00:01:33.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:33 smithi102 ceph-mon[34458]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 53 MiB/s wr, 22 op/s 2024-09-16T00:01:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:35 smithi031 ceph-mon[31203]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-16T00:01:35.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:35 smithi081 ceph-mon[35060]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-16T00:01:35.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:35 smithi102 ceph-mon[34458]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-16T00:01:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:37 smithi031 ceph-mon[31203]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 47 MiB/s wr, 22 op/s 2024-09-16T00:01:37.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:37 smithi081 ceph-mon[35060]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 47 MiB/s wr, 22 op/s 2024-09-16T00:01:37.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:37 smithi102 ceph-mon[34458]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 47 MiB/s wr, 22 op/s 2024-09-16T00:01:39.085 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:39.085 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 rewriters = 119223.60 kB/sec 2024-09-16T00:01:39.086 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 rewriters = 119220.50 kB/sec 2024-09-16T00:01:39.086 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 119223.60 kB/sec 2024-09-16T00:01:39.086 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 119223.60 kB/sec 2024-09-16T00:01:39.086 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 119223.60 kB/sec 2024-09-16T00:01:39.086 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:39 smithi031 ceph-mon[31203]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 39 MiB/s wr, 19 op/s 2024-09-16T00:01:39.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:39 smithi081 ceph-mon[35060]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 39 MiB/s wr, 19 op/s 2024-09-16T00:01:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:39 smithi102 ceph-mon[34458]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 39 MiB/s wr, 19 op/s 2024-09-16T00:01:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:41 smithi031 ceph-mon[31203]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 88 MiB/s wr, 36 op/s 2024-09-16T00:01:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:41 smithi031 ceph-mon[31203]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-16T00:01:41.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:41 smithi081 ceph-mon[35060]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 88 MiB/s wr, 36 op/s 2024-09-16T00:01:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:41 smithi081 ceph-mon[35060]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-16T00:01:41.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:41 smithi102 ceph-mon[34458]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 88 MiB/s wr, 36 op/s 2024-09-16T00:01:41.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:41 smithi102 ceph-mon[34458]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 readers = 7114970.50 kB/sec 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 readers = 7105188.75 kB/sec 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 7114970.50 kB/sec 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 7114970.50 kB/sec 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 7114970.50 kB/sec 2024-09-16T00:01:42.936 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:43.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:43 smithi031 ceph-mon[31203]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-16T00:01:43.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:43 smithi081 ceph-mon[35060]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-16T00:01:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:43 smithi102 ceph-mon[34458]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 re-readers = 7378879.00 kB/sec 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 re-readers = 7367013.20 kB/sec 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 7378879.00 kB/sec 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 7378879.00 kB/sec 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 7378879.00 kB/sec 2024-09-16T00:01:45.172 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:45 smithi031 ceph-mon[31203]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 35 op/s 2024-09-16T00:01:45.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:45 smithi081 ceph-mon[35060]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 35 op/s 2024-09-16T00:01:45.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:45 smithi102 ceph-mon[34458]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 35 op/s 2024-09-16T00:01:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:47 smithi031 ceph-mon[31203]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 57 MiB/s wr, 24 op/s 2024-09-16T00:01:47.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:47 smithi081 ceph-mon[35060]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 57 MiB/s wr, 24 op/s 2024-09-16T00:01:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:47 smithi102 ceph-mon[34458]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 57 MiB/s wr, 24 op/s 2024-09-16T00:01:49.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:49 smithi031 ceph-mon[31203]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 55 MiB/s wr, 23 op/s 2024-09-16T00:01:49.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:49 smithi081 ceph-mon[35060]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 55 MiB/s wr, 23 op/s 2024-09-16T00:01:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:49 smithi102 ceph-mon[34458]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 55 MiB/s wr, 23 op/s 2024-09-16T00:01:51.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:51 smithi031 ceph-mon[31203]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 55 MiB/s wr, 27 op/s 2024-09-16T00:01:51.520 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.520 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.520 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.521 INFO:tasks.workunit.client.0.smithi031.stdout:iozone test complete. 2024-09-16T00:01:51.521 INFO:tasks.workunit.client.0.smithi031.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-16T00:01:51.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:51 smithi081 ceph-mon[35060]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 55 MiB/s wr, 27 op/s 2024-09-16T00:01:51.588 INFO:tasks.workunit.client.0.smithi031.stdout: Iozone: Performance Test of File I/O 2024-09-16T00:01:51.588 INFO:tasks.workunit.client.0.smithi031.stdout: Version $Revision: 3.494 $ 2024-09-16T00:01:51.588 INFO:tasks.workunit.client.0.smithi031.stdout: Compiled for 64 bit mode. 2024-09-16T00:01:51.588 INFO:tasks.workunit.client.0.smithi031.stdout: Build: linux-AMD64 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: Run began: Mon Sep 16 00:01:51 2024 2024-09-16T00:01:51.589 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Include close in write timing 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Include fsync in write timing 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: File size set to 1048576 kB 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Record Size 1024 kB 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Output is in kBytes/sec 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache line size set to 32 bytes. 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: File stride size set to 17 * record size. 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Throughput test with 1 process 2024-09-16T00:01:51.590 INFO:tasks.workunit.client.0.smithi031.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-16T00:01:51.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:51 smithi102 ceph-mon[34458]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 55 MiB/s wr, 27 op/s 2024-09-16T00:01:53.182 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:53.182 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 initial writers = 672774.25 kB/sec 2024-09-16T00:01:53.182 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 initial writers = 672676.02 kB/sec 2024-09-16T00:01:53.182 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 672774.25 kB/sec 2024-09-16T00:01:53.182 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 672774.25 kB/sec 2024-09-16T00:01:53.183 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 672774.25 kB/sec 2024-09-16T00:01:53.183 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:53 smithi031 ceph-mon[31203]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-09-16T00:01:53.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:53 smithi081 ceph-mon[35060]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-09-16T00:01:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:53 smithi102 ceph-mon[34458]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-09-16T00:01:55.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:55 smithi031 ceph-mon[31203]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 56 op/s 2024-09-16T00:01:55.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:55 smithi081 ceph-mon[35060]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 56 op/s 2024-09-16T00:01:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:55 smithi102 ceph-mon[34458]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 56 op/s 2024-09-16T00:01:57.171 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:57.171 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 rewriters = 767705.06 kB/sec 2024-09-16T00:01:57.171 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 rewriters = 767583.58 kB/sec 2024-09-16T00:01:57.172 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 767705.06 kB/sec 2024-09-16T00:01:57.172 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 767705.06 kB/sec 2024-09-16T00:01:57.172 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 767705.06 kB/sec 2024-09-16T00:01:57.172 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:01:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:57 smithi031 ceph-mon[31203]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:57.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:57 smithi081 ceph-mon[35060]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:57 smithi102 ceph-mon[34458]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:01:59 smithi081 ceph-mon[35060]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:59.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:01:59 smithi102 ceph-mon[34458]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:01:59 smithi031 ceph-mon[31203]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 readers = 7634055.50 kB/sec 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 readers = 7621130.98 kB/sec 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 7634055.50 kB/sec 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 7634055.50 kB/sec 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 7634055.50 kB/sec 2024-09-16T00:01:59.976 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:02:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:01 smithi081 ceph-mon[35060]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-16T00:02:01.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:01 smithi102 ceph-mon[34458]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-16T00:02:01.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:01 smithi031 ceph-mon[31203]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 re-readers = 7681871.00 kB/sec 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 re-readers = 7666378.47 kB/sec 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 7681871.00 kB/sec 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 7681871.00 kB/sec 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 7681871.00 kB/sec 2024-09-16T00:02:02.192 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 1048576.00 kB 2024-09-16T00:02:03.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:03 smithi081 ceph-mon[35060]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-16T00:02:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:03 smithi102 ceph-mon[34458]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-16T00:02:03.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:03 smithi031 ceph-mon[31203]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 75 op/s 2024-09-16T00:02:05.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:05 smithi081 ceph-mon[35060]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-09-16T00:02:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:05 smithi102 ceph-mon[34458]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-09-16T00:02:05.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:05 smithi031 ceph-mon[31203]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-09-16T00:02:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:07 smithi081 ceph-mon[35060]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 90 MiB/s wr, 48 op/s 2024-09-16T00:02:07.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:07 smithi102 ceph-mon[34458]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 90 MiB/s wr, 48 op/s 2024-09-16T00:02:07.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:07 smithi031 ceph-mon[31203]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 90 MiB/s wr, 48 op/s 2024-09-16T00:02:08.972 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:08.972 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:08.972 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:08.972 INFO:tasks.workunit.client.0.smithi031.stdout:iozone test complete. 2024-09-16T00:02:08.973 INFO:tasks.workunit.client.0.smithi031.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-16T00:02:09.040 INFO:tasks.workunit.client.0.smithi031.stdout: Iozone: Performance Test of File I/O 2024-09-16T00:02:09.040 INFO:tasks.workunit.client.0.smithi031.stdout: Version $Revision: 3.494 $ 2024-09-16T00:02:09.040 INFO:tasks.workunit.client.0.smithi031.stdout: Compiled for 64 bit mode. 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Build: linux-AMD64 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: Run began: Mon Sep 16 00:02:08 2024 2024-09-16T00:02:09.041 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Include close in write timing 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Include fsync in write timing 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: File size set to 10485760 kB 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Record Size 1024 kB 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Output is in kBytes/sec 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Time Resolution = 0.000001 seconds. 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache size set to 1024 kBytes. 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Processor cache line size set to 32 bytes. 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: File stride size set to 17 * record size. 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Throughput test with 1 process 2024-09-16T00:02:09.042 INFO:tasks.workunit.client.0.smithi031.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-16T00:02:09.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:09 smithi081 ceph-mon[35060]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-09-16T00:02:09.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:09 smithi102 ceph-mon[34458]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-09-16T00:02:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:09 smithi031 ceph-mon[31203]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 85 MiB/s wr, 45 op/s 2024-09-16T00:02:11.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:11 smithi081 ceph-mon[35060]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 71 op/s 2024-09-16T00:02:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:11 smithi102 ceph-mon[34458]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 71 op/s 2024-09-16T00:02:11.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:11 smithi031 ceph-mon[31203]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 71 op/s 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 45 op/s 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:13 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 45 op/s 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:13 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 45 op/s 2024-09-16T00:02:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:02:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:13 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:02:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:02:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:14.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:02:14.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:02:14.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:14.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:02:14.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:02:14.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:02:15.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:15 smithi081 ceph-mon[35060]: pgmap v68: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 245 MiB/s wr, 113 op/s 2024-09-16T00:02:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:15 smithi102 ceph-mon[34458]: pgmap v68: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 245 MiB/s wr, 113 op/s 2024-09-16T00:02:15.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:15 smithi031 ceph-mon[31203]: pgmap v68: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 245 MiB/s wr, 113 op/s 2024-09-16T00:02:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:17 smithi081 ceph-mon[35060]: pgmap v69: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 95 op/s 2024-09-16T00:02:17.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:17 smithi102 ceph-mon[34458]: pgmap v69: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 95 op/s 2024-09-16T00:02:17.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:17 smithi031 ceph-mon[31203]: pgmap v69: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 MiB/s wr, 95 op/s 2024-09-16T00:02:19.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:19 smithi081 ceph-mon[35060]: pgmap v70: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 93 op/s 2024-09-16T00:02:19.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:19 smithi102 ceph-mon[34458]: pgmap v70: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 93 op/s 2024-09-16T00:02:19.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:19 smithi031 ceph-mon[31203]: pgmap v70: 97 pgs: 97 active+clean; 3.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 245 MiB/s wr, 93 op/s 2024-09-16T00:02:21.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:21 smithi081 ceph-mon[35060]: pgmap v71: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 527 MiB/s wr, 168 op/s 2024-09-16T00:02:21.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:21 smithi102 ceph-mon[34458]: pgmap v71: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 527 MiB/s wr, 168 op/s 2024-09-16T00:02:21.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:21 smithi031 ceph-mon[31203]: pgmap v71: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 527 MiB/s wr, 168 op/s 2024-09-16T00:02:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:23 smithi081 ceph-mon[35060]: pgmap v72: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 527 MiB/s wr, 142 op/s 2024-09-16T00:02:23.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:23 smithi102 ceph-mon[34458]: pgmap v72: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 527 MiB/s wr, 142 op/s 2024-09-16T00:02:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:23 smithi031 ceph-mon[31203]: pgmap v72: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 527 MiB/s wr, 142 op/s 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 initial writers = 684510.81 kB/sec 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 initial writers = 684500.33 kB/sec 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 684510.81 kB/sec 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 684510.81 kB/sec 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 684510.81 kB/sec 2024-09-16T00:02:24.393 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 10485760.00 kB 2024-09-16T00:02:25.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:25 smithi081 ceph-mon[35060]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 805 MiB/s wr, 216 op/s 2024-09-16T00:02:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:25 smithi102 ceph-mon[34458]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 805 MiB/s wr, 216 op/s 2024-09-16T00:02:25.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:25 smithi031 ceph-mon[31203]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 805 MiB/s wr, 216 op/s 2024-09-16T00:02:27.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:27 smithi081 ceph-mon[35060]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 559 MiB/s wr, 149 op/s 2024-09-16T00:02:27.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:27 smithi102 ceph-mon[34458]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 559 MiB/s wr, 149 op/s 2024-09-16T00:02:27.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:27 smithi031 ceph-mon[31203]: pgmap v74: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 559 MiB/s wr, 149 op/s 2024-09-16T00:02:29.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:29 smithi102 ceph-mon[34458]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 560 MiB/s wr, 149 op/s 2024-09-16T00:02:29.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:29 smithi031 ceph-mon[31203]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 560 MiB/s wr, 149 op/s 2024-09-16T00:02:29.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:29 smithi081 ceph-mon[35060]: pgmap v75: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 560 MiB/s wr, 149 op/s 2024-09-16T00:02:31.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:31 smithi102 ceph-mon[34458]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 608 MiB/s wr, 165 op/s 2024-09-16T00:02:31.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:31 smithi031 ceph-mon[31203]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 608 MiB/s wr, 165 op/s 2024-09-16T00:02:31.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:31 smithi081 ceph-mon[35060]: pgmap v76: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 608 MiB/s wr, 165 op/s 2024-09-16T00:02:33.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:33 smithi102 ceph-mon[34458]: pgmap v77: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 326 MiB/s wr, 90 op/s 2024-09-16T00:02:33.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:33 smithi031 ceph-mon[31203]: pgmap v77: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 326 MiB/s wr, 90 op/s 2024-09-16T00:02:33.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:33 smithi081 ceph-mon[35060]: pgmap v77: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 326 MiB/s wr, 90 op/s 2024-09-16T00:02:35.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:35 smithi102 ceph-mon[34458]: pgmap v78: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 590 MiB/s wr, 160 op/s 2024-09-16T00:02:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:35 smithi031 ceph-mon[31203]: pgmap v78: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 590 MiB/s wr, 160 op/s 2024-09-16T00:02:35.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:35 smithi081 ceph-mon[35060]: pgmap v78: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 590 MiB/s wr, 160 op/s 2024-09-16T00:02:37.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:37 smithi102 ceph-mon[34458]: pgmap v79: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:37 smithi031 ceph-mon[31203]: pgmap v79: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:37.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:37 smithi081 ceph-mon[35060]: pgmap v79: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:39 smithi102 ceph-mon[34458]: pgmap v80: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:39 smithi031 ceph-mon[31203]: pgmap v80: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:39.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:39 smithi081 ceph-mon[35060]: pgmap v80: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 312 MiB/s wr, 86 op/s 2024-09-16T00:02:41.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:41 smithi102 ceph-mon[34458]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 635 MiB/s wr, 171 op/s 2024-09-16T00:02:41.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:41 smithi031 ceph-mon[31203]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 635 MiB/s wr, 171 op/s 2024-09-16T00:02:41.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:41 smithi081 ceph-mon[35060]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 635 MiB/s wr, 171 op/s 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 rewriters = 787881.06 kB/sec 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 rewriters = 787866.31 kB/sec 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 787881.06 kB/sec 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 787881.06 kB/sec 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 787881.06 kB/sec 2024-09-16T00:02:42.673 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 10485760.00 kB 2024-09-16T00:02:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:43 smithi102 ceph-mon[34458]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 587 MiB/s wr, 155 op/s 2024-09-16T00:02:43.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:43 smithi031 ceph-mon[31203]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 587 MiB/s wr, 155 op/s 2024-09-16T00:02:43.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:43 smithi081 ceph-mon[35060]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 587 MiB/s wr, 155 op/s 2024-09-16T00:02:45.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:45 smithi102 ceph-mon[34458]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 853 MiB/s wr, 226 op/s 2024-09-16T00:02:45.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:45 smithi031 ceph-mon[31203]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 853 MiB/s wr, 226 op/s 2024-09-16T00:02:45.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:45 smithi081 ceph-mon[35060]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 853 MiB/s wr, 226 op/s 2024-09-16T00:02:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:47 smithi102 ceph-mon[34458]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:47.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:47 smithi031 ceph-mon[31203]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:47.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:47 smithi081 ceph-mon[35060]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:49 smithi102 ceph-mon[34458]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:49.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:49 smithi031 ceph-mon[31203]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:49.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:49 smithi081 ceph-mon[35060]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 589 MiB/s wr, 155 op/s 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 readers = 8997611.00 kB/sec 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 readers = 8995601.42 kB/sec 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 8997611.00 kB/sec 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 8997611.00 kB/sec 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 8997611.00 kB/sec 2024-09-16T00:02:50.980 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 10485760.00 kB 2024-09-16T00:02:51.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:51 smithi102 ceph-mon[34458]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 589 MiB/s wr, 160 op/s 2024-09-16T00:02:51.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:51 smithi031 ceph-mon[31203]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 589 MiB/s wr, 160 op/s 2024-09-16T00:02:51.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:51 smithi081 ceph-mon[35060]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 589 MiB/s wr, 160 op/s 2024-09-16T00:02:53.609 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:53 smithi081 ceph-mon[35060]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 267 MiB/s wr, 75 op/s 2024-09-16T00:02:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:53 smithi102 ceph-mon[34458]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 267 MiB/s wr, 75 op/s 2024-09-16T00:02:53.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:53 smithi031 ceph-mon[31203]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 267 MiB/s wr, 75 op/s 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Children see throughput for 1 re-readers = 8982540.00 kB/sec 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Parent sees throughput for 1 re-readers = 8981039.93 kB/sec 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Min throughput per process = 8982540.00 kB/sec 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Max throughput per process = 8982540.00 kB/sec 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Avg throughput per process = 8982540.00 kB/sec 2024-09-16T00:02:54.231 INFO:tasks.workunit.client.0.smithi031.stdout: Min xfer = 10485760.00 kB 2024-09-16T00:02:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:55 smithi102 ceph-mon[34458]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-09-16T00:02:55.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:55 smithi031 ceph-mon[31203]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-09-16T00:02:55.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:55 smithi081 ceph-mon[35060]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 267 MiB/s wr, 79 op/s 2024-09-16T00:02:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:57 smithi102 ceph-mon[34458]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:02:57.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:57 smithi031 ceph-mon[31203]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:02:57.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:57 smithi081 ceph-mon[35060]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:02:59.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:02:59 smithi102 ceph-mon[34458]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:02:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:02:59 smithi031 ceph-mon[31203]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:02:59.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:02:59 smithi081 ceph-mon[35060]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:03:01.089 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:03:01.090 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:03:01.090 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-09-16T00:03:01.090 INFO:tasks.workunit.client.0.smithi031.stdout:iozone test complete. 2024-09-16T00:03:01.090 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:03:01.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:03:01.171 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-16T00:03:01.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:03:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:01 smithi031 ceph-mon[31203]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:03:01.530 DEBUG:teuthology.parallel:result is None 2024-09-16T00:03:01.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:03:01.562 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:03:01.562 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-16T00:03:01.644 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-16T00:03:01.644 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:03:01.644 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:03:01.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-16T00:03:01.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:01 smithi102 ceph-mon[34458]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:03:01.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-16T00:03:01.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:01 smithi081 ceph-mon[35060]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:03:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:03 smithi102 ceph-mon[34458]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:03:03.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:03 smithi031 ceph-mon[31203]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:03:03.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:03 smithi081 ceph-mon[35060]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-16T00:03:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:05 smithi102 ceph-mon[34458]: pgmap v93: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:05.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:05 smithi031 ceph-mon[31203]: pgmap v93: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:05.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:05 smithi081 ceph-mon[35060]: pgmap v93: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:07.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:07 smithi031 ceph-mon[31203]: pgmap v94: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:07.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:07 smithi081 ceph-mon[35060]: pgmap v94: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:07.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:07 smithi102 ceph-mon[34458]: pgmap v94: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:09 smithi031 ceph-mon[31203]: pgmap v95: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:09.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:09 smithi081 ceph-mon[35060]: pgmap v95: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:09.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:09 smithi102 ceph-mon[34458]: pgmap v95: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 237 op/s 2024-09-16T00:03:11.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:11 smithi031 ceph-mon[31203]: pgmap v96: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 242 op/s 2024-09-16T00:03:11.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:11 smithi081 ceph-mon[35060]: pgmap v96: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 242 op/s 2024-09-16T00:03:11.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:11 smithi102 ceph-mon[34458]: pgmap v96: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 242 op/s 2024-09-16T00:03:13.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:13 smithi031 ceph-mon[31203]: pgmap v97: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 238 op/s 2024-09-16T00:03:13.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:13 smithi081 ceph-mon[35060]: pgmap v97: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 238 op/s 2024-09-16T00:03:13.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:13 smithi102 ceph-mon[34458]: pgmap v97: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 238 op/s 2024-09-16T00:03:14.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:14.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:03:14.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:03:14.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:03:14.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:14.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:03:14.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:03:14.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:03:14.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:14.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:03:14.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:03:14.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:03:15.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:15 smithi031 ceph-mon[31203]: pgmap v98: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:15.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:15 smithi081 ceph-mon[35060]: pgmap v98: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:15.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:15 smithi102 ceph-mon[34458]: pgmap v98: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 242 op/s 2024-09-16T00:03:17.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:17 smithi031 ceph-mon[31203]: pgmap v99: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:17.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:17 smithi081 ceph-mon[35060]: pgmap v99: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:17.873 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:17 smithi102 ceph-mon[34458]: pgmap v99: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:19.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:19 smithi031 ceph-mon[31203]: pgmap v100: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:19.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:19 smithi081 ceph-mon[35060]: pgmap v100: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:19.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:19 smithi102 ceph-mon[34458]: pgmap v100: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-16T00:03:21.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:21 smithi031 ceph-mon[31203]: pgmap v101: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-16T00:03:21.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:21 smithi081 ceph-mon[35060]: pgmap v101: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-16T00:03:21.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:21 smithi102 ceph-mon[34458]: pgmap v101: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-16T00:03:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:23 smithi031 ceph-mon[31203]: pgmap v102: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T00:03:23.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:23 smithi081 ceph-mon[35060]: pgmap v102: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T00:03:23.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:23 smithi102 ceph-mon[34458]: pgmap v102: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-16T00:03:25.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:25 smithi031 ceph-mon[31203]: pgmap v103: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-16T00:03:25.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:25 smithi081 ceph-mon[35060]: pgmap v103: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-16T00:03:25.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:25 smithi102 ceph-mon[34458]: pgmap v103: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-16T00:03:26.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:26 smithi031 ceph-mon[31203]: pgmap v104: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:26.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:26 smithi081 ceph-mon[35060]: pgmap v104: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:26.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:26 smithi102 ceph-mon[34458]: pgmap v104: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:28.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:28 smithi102 ceph-mon[34458]: pgmap v105: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:29.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:28 smithi031 ceph-mon[31203]: pgmap v105: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:29.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:28 smithi081 ceph-mon[35060]: pgmap v105: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:30 smithi031 ceph-mon[31203]: pgmap v106: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:30 smithi081 ceph-mon[35060]: pgmap v106: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:31.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:30 smithi102 ceph-mon[34458]: pgmap v106: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:33.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:32 smithi031 ceph-mon[31203]: pgmap v107: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:32 smithi081 ceph-mon[35060]: pgmap v107: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:33.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:32 smithi102 ceph-mon[34458]: pgmap v107: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:35.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:34 smithi031 ceph-mon[31203]: pgmap v108: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:35.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:34 smithi081 ceph-mon[35060]: pgmap v108: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:35.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:34 smithi102 ceph-mon[34458]: pgmap v108: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:37.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:36 smithi031 ceph-mon[31203]: pgmap v109: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:37.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:36 smithi081 ceph-mon[35060]: pgmap v109: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:37.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:36 smithi102 ceph-mon[34458]: pgmap v109: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:38 smithi031 ceph-mon[31203]: pgmap v110: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:39.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:38 smithi081 ceph-mon[35060]: pgmap v110: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:39.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:38 smithi102 ceph-mon[34458]: pgmap v110: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:41.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:40 smithi031 ceph-mon[31203]: pgmap v111: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:41.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:40 smithi081 ceph-mon[35060]: pgmap v111: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:41.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:40 smithi102 ceph-mon[34458]: pgmap v111: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:43.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:42 smithi031 ceph-mon[31203]: pgmap v112: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:43.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:42 smithi081 ceph-mon[35060]: pgmap v112: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:43.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:42 smithi102 ceph-mon[34458]: pgmap v112: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:45.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:44 smithi031 ceph-mon[31203]: pgmap v113: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:45.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:44 smithi081 ceph-mon[35060]: pgmap v113: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:44 smithi102 ceph-mon[34458]: pgmap v113: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:47.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:46 smithi031 ceph-mon[31203]: pgmap v114: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:47.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:46 smithi081 ceph-mon[35060]: pgmap v114: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:47.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:46 smithi102 ceph-mon[34458]: pgmap v114: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:49.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:48 smithi031 ceph-mon[31203]: pgmap v115: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:49.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:48 smithi081 ceph-mon[35060]: pgmap v115: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:49.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:48 smithi102 ceph-mon[34458]: pgmap v115: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:51.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:50 smithi031 ceph-mon[31203]: pgmap v116: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:51.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:50 smithi081 ceph-mon[35060]: pgmap v116: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:50 smithi102 ceph-mon[34458]: pgmap v116: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:53.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:52 smithi031 ceph-mon[31203]: pgmap v117: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:53.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:52 smithi081 ceph-mon[35060]: pgmap v117: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:53.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:52 smithi102 ceph-mon[34458]: pgmap v117: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:55.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:54 smithi031 ceph-mon[31203]: pgmap v118: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:55.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:54 smithi081 ceph-mon[35060]: pgmap v118: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:55.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:54 smithi102 ceph-mon[34458]: pgmap v118: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:03:57.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:56 smithi031 ceph-mon[31203]: pgmap v119: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:57.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:56 smithi081 ceph-mon[35060]: pgmap v119: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:57.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:56 smithi102 ceph-mon[34458]: pgmap v119: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:58.923 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:03:58 smithi102 ceph-mon[34458]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:59.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:03:58 smithi031 ceph-mon[31203]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:03:59.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:03:58 smithi081 ceph-mon[35060]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:01.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:00 smithi031 ceph-mon[31203]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:01.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:00 smithi081 ceph-mon[35060]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:01.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:00 smithi102 ceph-mon[34458]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:03.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:02 smithi031 ceph-mon[31203]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:03.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:02 smithi081 ceph-mon[35060]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:03.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:02 smithi102 ceph-mon[34458]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:04 smithi031 ceph-mon[31203]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:04 smithi081 ceph-mon[35060]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:04 smithi102 ceph-mon[34458]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:07.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:06 smithi031 ceph-mon[31203]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:07.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:06 smithi081 ceph-mon[35060]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:07.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:06 smithi102 ceph-mon[34458]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:09.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:08 smithi031 ceph-mon[31203]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:09.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:08 smithi081 ceph-mon[35060]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:08 smithi102 ceph-mon[34458]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:11.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:10 smithi031 ceph-mon[31203]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:10 smithi081 ceph-mon[35060]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:11.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:10 smithi102 ceph-mon[34458]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:12 smithi031 ceph-mon[31203]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:13.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:12 smithi081 ceph-mon[35060]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:12 smithi102 ceph-mon[34458]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:04:15.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:14 smithi031 ceph-mon[31203]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:15.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:14 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:15.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:14 smithi081 ceph-mon[35060]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:15.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:14 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:15.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:14 smithi102 ceph-mon[34458]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:04:15.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:14 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:16.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:16.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:16.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:04:16.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:16.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:16.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:04:16.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:16.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:16.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:04:17.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:16 smithi031 ceph-mon[31203]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:17.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:16 smithi081 ceph-mon[35060]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:17.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:16 smithi102 ceph-mon[34458]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:19.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:18 smithi031 ceph-mon[31203]: pgmap v130: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:19.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:18 smithi081 ceph-mon[35060]: pgmap v130: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:19.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:18 smithi102 ceph-mon[34458]: pgmap v130: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:04:21.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:20 smithi031 ceph-mon[31203]: pgmap v131: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:04:21.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:20 smithi081 ceph-mon[35060]: pgmap v131: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:04:21.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:20 smithi102 ceph-mon[34458]: pgmap v131: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:04:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:22 smithi031 ceph-mon[31203]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:23.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:22 smithi081 ceph-mon[35060]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:23.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:22 smithi102 ceph-mon[34458]: pgmap v132: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:25.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:24 smithi031 ceph-mon[31203]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:25.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:24 smithi081 ceph-mon[35060]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:25.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:24 smithi102 ceph-mon[34458]: pgmap v133: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:27.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:26 smithi031 ceph-mon[31203]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:26 smithi081 ceph-mon[35060]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:27.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:26 smithi102 ceph-mon[34458]: pgmap v134: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:29.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:28 smithi031 ceph-mon[31203]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:29.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:28 smithi081 ceph-mon[35060]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:29.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:28 smithi102 ceph-mon[34458]: pgmap v135: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:31.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:30 smithi031 ceph-mon[31203]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:31.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:30 smithi081 ceph-mon[35060]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:31.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:30 smithi102 ceph-mon[34458]: pgmap v136: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:33.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:32 smithi031 ceph-mon[31203]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:33.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:32 smithi081 ceph-mon[35060]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:33.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:32 smithi102 ceph-mon[34458]: pgmap v137: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:35.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:34 smithi031 ceph-mon[31203]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:35.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:34 smithi081 ceph-mon[35060]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:35.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:34 smithi102 ceph-mon[34458]: pgmap v138: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:37.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:36 smithi031 ceph-mon[31203]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:37.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:36 smithi081 ceph-mon[35060]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:37.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:36 smithi102 ceph-mon[34458]: pgmap v139: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:39.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:38 smithi031 ceph-mon[31203]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:39.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:38 smithi081 ceph-mon[35060]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:39.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:38 smithi102 ceph-mon[34458]: pgmap v140: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:41.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:40 smithi031 ceph-mon[31203]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:41.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:40 smithi081 ceph-mon[35060]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:41.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:40 smithi102 ceph-mon[34458]: pgmap v141: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:43.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:42 smithi081 ceph-mon[35060]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:43.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:42 smithi102 ceph-mon[34458]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:43.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:42 smithi031 ceph-mon[31203]: pgmap v142: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:45.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:44 smithi081 ceph-mon[35060]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:45.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:44 smithi031 ceph-mon[31203]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:45.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:44 smithi102 ceph-mon[34458]: pgmap v143: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:47.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:46 smithi081 ceph-mon[35060]: pgmap v144: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:47.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:46 smithi102 ceph-mon[34458]: pgmap v144: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:47.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:46 smithi031 ceph-mon[31203]: pgmap v144: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:49.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:48 smithi081 ceph-mon[35060]: pgmap v145: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:49.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:48 smithi102 ceph-mon[34458]: pgmap v145: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:49.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:48 smithi031 ceph-mon[31203]: pgmap v145: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:51.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:50 smithi081 ceph-mon[35060]: pgmap v146: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:51.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:50 smithi102 ceph-mon[34458]: pgmap v146: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:51.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:50 smithi031 ceph-mon[31203]: pgmap v146: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:53.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:52 smithi081 ceph-mon[35060]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:53.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:52 smithi102 ceph-mon[34458]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:53.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:52 smithi031 ceph-mon[31203]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:55.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:54 smithi081 ceph-mon[35060]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:55.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:54 smithi102 ceph-mon[34458]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:55.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:54 smithi031 ceph-mon[31203]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:04:57.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:56 smithi081 ceph-mon[35060]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:57.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:56 smithi102 ceph-mon[34458]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:57.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:56 smithi031 ceph-mon[31203]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:59.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:04:58 smithi081 ceph-mon[35060]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:59.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:04:58 smithi102 ceph-mon[34458]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:04:59.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:04:58 smithi031 ceph-mon[31203]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:01.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:00 smithi081 ceph-mon[35060]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:01.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:00 smithi102 ceph-mon[34458]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:01.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:00 smithi031 ceph-mon[31203]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:03.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:02 smithi081 ceph-mon[35060]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:03.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:02 smithi102 ceph-mon[34458]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:02 smithi031 ceph-mon[31203]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:05.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:04 smithi081 ceph-mon[35060]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:04 smithi102 ceph-mon[34458]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:05.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:04 smithi031 ceph-mon[31203]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:07.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:06 smithi081 ceph-mon[35060]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:07.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:06 smithi102 ceph-mon[34458]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:06 smithi031 ceph-mon[31203]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:09.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:08 smithi081 ceph-mon[35060]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:08 smithi102 ceph-mon[34458]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:09.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:08 smithi031 ceph-mon[31203]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:10 smithi081 ceph-mon[35060]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:11.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:10 smithi102 ceph-mon[34458]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:11.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:10 smithi031 ceph-mon[31203]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:13.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:12 smithi081 ceph-mon[35060]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:12 smithi102 ceph-mon[34458]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:12 smithi031 ceph-mon[31203]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:05:15.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:14 smithi031 ceph-mon[31203]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:15.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:14 smithi081 ceph-mon[35060]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:15.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:14 smithi102 ceph-mon[34458]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:05:16.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:16.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:16.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:16.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:15 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:05:16.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:16.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:16.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:16.121 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:15 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:05:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:15 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:05:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:17 smithi081 ceph-mon[35060]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:17.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:17 smithi102 ceph-mon[34458]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:17.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:17 smithi031 ceph-mon[31203]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:19.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:19 smithi081 ceph-mon[35060]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:19.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:19 smithi102 ceph-mon[34458]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:19.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:19 smithi031 ceph-mon[31203]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:05:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:21 smithi081 ceph-mon[35060]: pgmap v161: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:05:21.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:21 smithi102 ceph-mon[34458]: pgmap v161: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:05:21.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:21 smithi031 ceph-mon[31203]: pgmap v161: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:05:23.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:23 smithi081 ceph-mon[35060]: pgmap v162: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:23.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:23 smithi102 ceph-mon[34458]: pgmap v162: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:23 smithi031 ceph-mon[31203]: pgmap v162: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:25.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:25 smithi081 ceph-mon[35060]: pgmap v163: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:25.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:25 smithi102 ceph-mon[34458]: pgmap v163: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:25.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:25 smithi031 ceph-mon[31203]: pgmap v163: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:27.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:27 smithi081 ceph-mon[35060]: pgmap v164: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:27.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:27 smithi102 ceph-mon[34458]: pgmap v164: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:27.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:27 smithi031 ceph-mon[31203]: pgmap v164: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:29.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:29 smithi081 ceph-mon[35060]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:29.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:29 smithi102 ceph-mon[34458]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:29.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:29 smithi031 ceph-mon[31203]: pgmap v165: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:31.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:31 smithi081 ceph-mon[35060]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:31.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:31 smithi102 ceph-mon[34458]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:31.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:31 smithi031 ceph-mon[31203]: pgmap v166: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:33.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:33 smithi081 ceph-mon[35060]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:05:33.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:33 smithi102 ceph-mon[34458]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:05:33.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:33 smithi031 ceph-mon[31203]: pgmap v167: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:05:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:35 smithi081 ceph-mon[35060]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:35.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:35 smithi102 ceph-mon[34458]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:35 smithi031 ceph-mon[31203]: pgmap v168: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:37.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:37 smithi081 ceph-mon[35060]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:37.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:37 smithi102 ceph-mon[34458]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:37 smithi031 ceph-mon[31203]: pgmap v169: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:39.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:39 smithi081 ceph-mon[35060]: pgmap v170: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:39.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:39 smithi102 ceph-mon[34458]: pgmap v170: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:39 smithi031 ceph-mon[31203]: pgmap v170: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-16T00:05:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:41 smithi081 ceph-mon[35060]: pgmap v171: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:41.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:41 smithi102 ceph-mon[34458]: pgmap v171: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:41.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:41 smithi031 ceph-mon[31203]: pgmap v171: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:05:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:43 smithi102 ceph-mon[34458]: pgmap v172: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:43.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:43 smithi031 ceph-mon[31203]: pgmap v172: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:43.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:43 smithi081 ceph-mon[35060]: pgmap v172: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:45.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:45 smithi102 ceph-mon[34458]: pgmap v173: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:45 smithi031 ceph-mon[31203]: pgmap v173: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:45.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:45 smithi081 ceph-mon[35060]: pgmap v173: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:47 smithi102 ceph-mon[34458]: pgmap v174: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:47.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:47 smithi031 ceph-mon[31203]: pgmap v174: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:47.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:47 smithi081 ceph-mon[35060]: pgmap v174: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:49 smithi102 ceph-mon[34458]: pgmap v175: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:49.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:49 smithi031 ceph-mon[31203]: pgmap v175: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:49.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:49 smithi081 ceph-mon[35060]: pgmap v175: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:51.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:51 smithi102 ceph-mon[34458]: pgmap v176: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:51.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:51 smithi031 ceph-mon[31203]: pgmap v176: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:51.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:51 smithi081 ceph-mon[35060]: pgmap v176: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:53 smithi102 ceph-mon[34458]: pgmap v177: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:53.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:53 smithi031 ceph-mon[31203]: pgmap v177: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:53.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:53 smithi081 ceph-mon[35060]: pgmap v177: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:55.609 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:55 smithi081 ceph-mon[35060]: pgmap v178: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:55 smithi102 ceph-mon[34458]: pgmap v178: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:55.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:55 smithi031 ceph-mon[31203]: pgmap v178: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:05:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:57 smithi102 ceph-mon[34458]: pgmap v179: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:57.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:57 smithi031 ceph-mon[31203]: pgmap v179: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:57.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:57 smithi081 ceph-mon[35060]: pgmap v179: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:59.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:05:59 smithi102 ceph-mon[34458]: pgmap v180: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:05:59 smithi031 ceph-mon[31203]: pgmap v180: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:05:59.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:05:59 smithi081 ceph-mon[35060]: pgmap v180: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:01 smithi102 ceph-mon[34458]: pgmap v181: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:06:01.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:01 smithi031 ceph-mon[31203]: pgmap v181: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:06:01.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:01 smithi081 ceph-mon[35060]: pgmap v181: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-16T00:06:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:03 smithi102 ceph-mon[34458]: pgmap v182: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:06:03.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:03 smithi031 ceph-mon[31203]: pgmap v182: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:06:03.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:03 smithi081 ceph-mon[35060]: pgmap v182: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:06:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:05 smithi102 ceph-mon[34458]: pgmap v183: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:05.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:05 smithi031 ceph-mon[31203]: pgmap v183: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:05.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:05 smithi081 ceph-mon[35060]: pgmap v183: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:07.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:07 smithi102 ceph-mon[34458]: pgmap v184: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:07.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:07 smithi031 ceph-mon[31203]: pgmap v184: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:07.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:07 smithi081 ceph-mon[35060]: pgmap v184: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:09.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:09 smithi102 ceph-mon[34458]: pgmap v185: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:09.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:09 smithi031 ceph-mon[31203]: pgmap v185: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:09.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:09 smithi081 ceph-mon[35060]: pgmap v185: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-16T00:06:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:11 smithi102 ceph-mon[34458]: pgmap v186: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:11.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:11 smithi031 ceph-mon[31203]: pgmap v186: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:11.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:11 smithi081 ceph-mon[35060]: pgmap v186: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:13 smithi102 ceph-mon[34458]: pgmap v187: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:06:13.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:13 smithi031 ceph-mon[31203]: pgmap v187: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:06:13.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:13 smithi081 ceph-mon[35060]: pgmap v187: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-16T00:06:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:15 smithi102 ceph-mon[34458]: pgmap v188: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:15.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:15 smithi031 ceph-mon[31203]: pgmap v188: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:15.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:15 smithi081 ceph-mon[35060]: pgmap v188: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-16T00:06:16.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:16 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:16.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:16 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:16.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:16 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:16.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:16 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:06:16.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:16 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:16.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:16 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:16.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:16 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:16.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:16 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:06:16.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:16 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:16.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:16 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:16.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:16 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:16.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:16 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:06:17.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:17 smithi102 ceph-mon[34458]: pgmap v189: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:17.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:17 smithi031 ceph-mon[31203]: pgmap v189: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:17.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:17 smithi081 ceph-mon[35060]: pgmap v189: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:19.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:19 smithi031 ceph-mon[31203]: pgmap v190: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:19.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:19 smithi081 ceph-mon[35060]: pgmap v190: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:19.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:19 smithi102 ceph-mon[34458]: pgmap v190: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-16T00:06:21.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:21 smithi031 ceph-mon[31203]: pgmap v191: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:06:21.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:21 smithi081 ceph-mon[35060]: pgmap v191: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:06:21.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:21 smithi102 ceph-mon[34458]: pgmap v191: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-16T00:06:23.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:23 smithi031 ceph-mon[31203]: pgmap v192: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:23.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:23 smithi081 ceph-mon[35060]: pgmap v192: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:23.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:23 smithi102 ceph-mon[34458]: pgmap v192: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:25.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:25 smithi031 ceph-mon[31203]: pgmap v193: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:25.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:25 smithi081 ceph-mon[35060]: pgmap v193: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:25.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:25 smithi102 ceph-mon[34458]: pgmap v193: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:27.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:27 smithi031 ceph-mon[31203]: pgmap v194: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:27.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:27 smithi081 ceph-mon[35060]: pgmap v194: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:27.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:27 smithi102 ceph-mon[34458]: pgmap v194: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:29.673 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:29 smithi102 ceph-mon[34458]: pgmap v195: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:29.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:29 smithi031 ceph-mon[31203]: pgmap v195: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:29.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:29 smithi081 ceph-mon[35060]: pgmap v195: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:31.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:31 smithi031 ceph-mon[31203]: pgmap v196: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:31.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:31 smithi081 ceph-mon[35060]: pgmap v196: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:31.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:31 smithi102 ceph-mon[34458]: pgmap v196: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:33.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:33 smithi031 ceph-mon[31203]: pgmap v197: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:33.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:33 smithi081 ceph-mon[35060]: pgmap v197: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:33.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:33 smithi102 ceph-mon[34458]: pgmap v197: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:35 smithi031 ceph-mon[31203]: pgmap v198: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:35.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:35 smithi081 ceph-mon[35060]: pgmap v198: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:35.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:35 smithi102 ceph-mon[34458]: pgmap v198: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:37.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:37 smithi031 ceph-mon[31203]: pgmap v199: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:37.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:37 smithi081 ceph-mon[35060]: pgmap v199: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:37.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:37 smithi102 ceph-mon[34458]: pgmap v199: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:39.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:39 smithi031 ceph-mon[31203]: pgmap v200: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:39.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:39 smithi081 ceph-mon[35060]: pgmap v200: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:39.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:39 smithi102 ceph-mon[34458]: pgmap v200: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:41.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:41 smithi031 ceph-mon[31203]: pgmap v201: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:41.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:41 smithi081 ceph-mon[35060]: pgmap v201: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:41.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:41 smithi102 ceph-mon[34458]: pgmap v201: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:43.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:43 smithi031 ceph-mon[31203]: pgmap v202: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:43.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:43 smithi081 ceph-mon[35060]: pgmap v202: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:43.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:43 smithi102 ceph-mon[34458]: pgmap v202: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:45 smithi031 ceph-mon[31203]: pgmap v203: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:45.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:45 smithi081 ceph-mon[35060]: pgmap v203: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:45.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:45 smithi102 ceph-mon[34458]: pgmap v203: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:47.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:47 smithi031 ceph-mon[31203]: pgmap v204: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:47.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:47 smithi081 ceph-mon[35060]: pgmap v204: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:47.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:47 smithi102 ceph-mon[34458]: pgmap v204: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:49.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:49 smithi031 ceph-mon[31203]: pgmap v205: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:49.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:49 smithi081 ceph-mon[35060]: pgmap v205: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:49.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:49 smithi102 ceph-mon[34458]: pgmap v205: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:51.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:51 smithi031 ceph-mon[31203]: pgmap v206: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:51.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:51 smithi081 ceph-mon[35060]: pgmap v206: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:51.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:51 smithi102 ceph-mon[34458]: pgmap v206: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:53.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:53 smithi031 ceph-mon[31203]: pgmap v207: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:53.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:53 smithi081 ceph-mon[35060]: pgmap v207: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:53.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:53 smithi102 ceph-mon[34458]: pgmap v207: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:55.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:55 smithi031 ceph-mon[31203]: pgmap v208: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:55.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:55 smithi081 ceph-mon[35060]: pgmap v208: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:55.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:55 smithi102 ceph-mon[34458]: pgmap v208: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:06:57.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:57 smithi031 ceph-mon[31203]: pgmap v209: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:57.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:57 smithi081 ceph-mon[35060]: pgmap v209: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:57.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:57 smithi102 ceph-mon[34458]: pgmap v209: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:06:59 smithi031 ceph-mon[31203]: pgmap v210: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:59.807 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:06:59 smithi081 ceph-mon[35060]: pgmap v210: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:06:59.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:06:59 smithi102 ceph-mon[34458]: pgmap v210: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:01.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:01 smithi031 ceph-mon[31203]: pgmap v211: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:01.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:01 smithi081 ceph-mon[35060]: pgmap v211: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:01.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:01 smithi102 ceph-mon[34458]: pgmap v211: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:02.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:02 smithi031 ceph-mon[31203]: pgmap v212: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:02.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:02 smithi081 ceph-mon[35060]: pgmap v212: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:02.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:02 smithi102 ceph-mon[34458]: pgmap v212: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:05.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:04 smithi031 ceph-mon[31203]: pgmap v213: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:07:05.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:04 smithi081 ceph-mon[35060]: pgmap v213: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:07:05.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:04 smithi102 ceph-mon[34458]: pgmap v213: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:07:07.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:06 smithi031 ceph-mon[31203]: pgmap v214: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:07:07.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:06 smithi081 ceph-mon[35060]: pgmap v214: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:07:07.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:06 smithi102 ceph-mon[34458]: pgmap v214: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:07:08.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:07 smithi031 ceph-mon[31203]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:07:08.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:07 smithi081 ceph-mon[35060]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:07:08.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:07 smithi102 ceph-mon[34458]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:07:09.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:08 smithi031 ceph-mon[31203]: pgmap v216: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:09.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:08 smithi081 ceph-mon[35060]: pgmap v216: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:09.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:08 smithi102 ceph-mon[34458]: pgmap v216: 97 pgs: 97 active+clean; 1.0 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:11.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:10 smithi031 ceph-mon[31203]: pgmap v217: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:11.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:10 smithi081 ceph-mon[35060]: pgmap v217: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:11.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:10 smithi102 ceph-mon[34458]: pgmap v217: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:12 smithi031 ceph-mon[31203]: pgmap v218: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:13.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:12 smithi081 ceph-mon[35060]: pgmap v218: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:13.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:12 smithi102 ceph-mon[34458]: pgmap v218: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:07:15.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:15 smithi081 ceph-mon[35060]: pgmap v219: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:15.307 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:15 smithi081 ceph-mon[35060]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:07:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:15 smithi102 ceph-mon[34458]: pgmap v219: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:15 smithi102 ceph-mon[34458]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:07:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:15 smithi031 ceph-mon[31203]: pgmap v219: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:15 smithi031 ceph-mon[31203]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:07:16.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:16 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:16.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:16 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:16.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:16 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:17 smithi102 ceph-mon[34458]: pgmap v221: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-16T00:07:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:07:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:17 smithi031 ceph-mon[31203]: pgmap v221: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-16T00:07:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:17.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:07:17.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:17 smithi081 ceph-mon[35060]: pgmap v221: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-09-16T00:07:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:07:19.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:19 smithi102 ceph-mon[34458]: pgmap v222: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:19 smithi031 ceph-mon[31203]: pgmap v222: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:19 smithi081 ceph-mon[35060]: pgmap v222: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:07:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:21 smithi102 ceph-mon[34458]: pgmap v223: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:21 smithi031 ceph-mon[31203]: pgmap v223: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:21 smithi081 ceph-mon[35060]: pgmap v223: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:23 smithi102 ceph-mon[34458]: pgmap v224: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:23 smithi031 ceph-mon[31203]: pgmap v224: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:23.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:23 smithi081 ceph-mon[35060]: pgmap v224: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:07:25.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:25 smithi081 ceph-mon[35060]: pgmap v225: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:07:25.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:25 smithi102 ceph-mon[34458]: pgmap v225: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:07:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:25 smithi031 ceph-mon[31203]: pgmap v225: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:07:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:27 smithi102 ceph-mon[34458]: pgmap v226: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:07:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:27 smithi031 ceph-mon[31203]: pgmap v226: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:07:27.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:27 smithi081 ceph-mon[35060]: pgmap v226: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:07:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:29 smithi102 ceph-mon[34458]: pgmap v227: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:29.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:29 smithi031 ceph-mon[31203]: pgmap v227: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:29.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:29 smithi081 ceph-mon[35060]: pgmap v227: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:31 smithi102 ceph-mon[34458]: pgmap v228: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:31 smithi031 ceph-mon[31203]: pgmap v228: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:31.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:31 smithi081 ceph-mon[35060]: pgmap v228: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:33.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:33 smithi102 ceph-mon[34458]: pgmap v229: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:33.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:33 smithi031 ceph-mon[31203]: pgmap v229: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:33 smithi081 ceph-mon[35060]: pgmap v229: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:35 smithi102 ceph-mon[34458]: pgmap v230: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:35 smithi031 ceph-mon[31203]: pgmap v230: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:35 smithi081 ceph-mon[35060]: pgmap v230: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:37 smithi102 ceph-mon[34458]: pgmap v231: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:37 smithi031 ceph-mon[31203]: pgmap v231: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:37.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:37 smithi081 ceph-mon[35060]: pgmap v231: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:39.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:39 smithi102 ceph-mon[34458]: pgmap v232: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:39 smithi031 ceph-mon[31203]: pgmap v232: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:39 smithi081 ceph-mon[35060]: pgmap v232: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:41 smithi102 ceph-mon[34458]: pgmap v233: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:41 smithi031 ceph-mon[31203]: pgmap v233: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:41 smithi081 ceph-mon[35060]: pgmap v233: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:43.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:43 smithi102 ceph-mon[34458]: pgmap v234: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:43.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:43 smithi031 ceph-mon[31203]: pgmap v234: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:43.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:43 smithi081 ceph-mon[35060]: pgmap v234: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:45.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:45 smithi102 ceph-mon[34458]: pgmap v235: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:45 smithi031 ceph-mon[31203]: pgmap v235: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:45.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:45 smithi081 ceph-mon[35060]: pgmap v235: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:47.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:47 smithi102 ceph-mon[34458]: pgmap v236: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:47 smithi031 ceph-mon[31203]: pgmap v236: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:47.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:47 smithi081 ceph-mon[35060]: pgmap v236: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:49.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:49 smithi102 ceph-mon[34458]: pgmap v237: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:49.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:49 smithi031 ceph-mon[31203]: pgmap v237: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:49.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:49 smithi081 ceph-mon[35060]: pgmap v237: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:51.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:51 smithi102 ceph-mon[34458]: pgmap v238: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:51.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:51 smithi031 ceph-mon[31203]: pgmap v238: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:51.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:51 smithi081 ceph-mon[35060]: pgmap v238: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:53.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:53 smithi102 ceph-mon[34458]: pgmap v239: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:53 smithi031 ceph-mon[31203]: pgmap v239: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:53.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:53 smithi081 ceph-mon[35060]: pgmap v239: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:55.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:55 smithi031 ceph-mon[31203]: pgmap v240: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:56.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:55 smithi102 ceph-mon[34458]: pgmap v240: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:56.134 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:55 smithi081 ceph-mon[35060]: pgmap v240: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:07:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:57 smithi031 ceph-mon[31203]: pgmap v241: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:57.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:57 smithi081 ceph-mon[35060]: pgmap v241: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:57 smithi102 ceph-mon[34458]: pgmap v241: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:59.423 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:07:59 smithi102 ceph-mon[34458]: pgmap v242: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:07:59 smithi031 ceph-mon[31203]: pgmap v242: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:07:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:07:59 smithi081 ceph-mon[35060]: pgmap v242: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:01 smithi031 ceph-mon[31203]: pgmap v243: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:01.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:01 smithi081 ceph-mon[35060]: pgmap v243: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:01 smithi102 ceph-mon[34458]: pgmap v243: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:03 smithi031 ceph-mon[31203]: pgmap v244: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:03.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:03 smithi081 ceph-mon[35060]: pgmap v244: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:03 smithi102 ceph-mon[34458]: pgmap v244: 97 pgs: 97 active+clean; 1.0 GiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:05.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:05 smithi031 ceph-mon[31203]: pgmap v245: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:08:05.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:05 smithi081 ceph-mon[35060]: pgmap v245: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:08:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:05 smithi102 ceph-mon[34458]: pgmap v245: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:08:07.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:07 smithi031 ceph-mon[31203]: pgmap v246: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:08:07.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:07 smithi081 ceph-mon[35060]: pgmap v246: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:08:07.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:07 smithi102 ceph-mon[34458]: pgmap v246: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:08:08.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:08 smithi031 ceph-mon[31203]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:08:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:08 smithi081 ceph-mon[35060]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:08:08.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:08 smithi102 ceph-mon[34458]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:08:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:09 smithi031 ceph-mon[31203]: pgmap v248: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:09.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:09 smithi081 ceph-mon[35060]: pgmap v248: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:09.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:09 smithi102 ceph-mon[34458]: pgmap v248: 97 pgs: 97 active+clean; 1.0 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:11 smithi031 ceph-mon[31203]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:11 smithi081 ceph-mon[35060]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:11 smithi102 ceph-mon[34458]: pgmap v249: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:13.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:13 smithi031 ceph-mon[31203]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:13 smithi081 ceph-mon[35060]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:13 smithi102 ceph-mon[34458]: pgmap v250: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:08:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:15 smithi102 ceph-mon[34458]: pgmap v251: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:15.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:15 smithi102 ceph-mon[34458]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:08:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:15 smithi031 ceph-mon[31203]: pgmap v251: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:15 smithi031 ceph-mon[31203]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:08:15.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:15 smithi081 ceph-mon[35060]: pgmap v251: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:15.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:15 smithi081 ceph-mon[35060]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:08:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:17 smithi102 ceph-mon[34458]: pgmap v253: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:08:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:17.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:08:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:17 smithi031 ceph-mon[31203]: pgmap v253: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:08:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:08:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:17 smithi081 ceph-mon[35060]: pgmap v253: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:08:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:08:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:19 smithi102 ceph-mon[34458]: pgmap v254: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:19 smithi031 ceph-mon[31203]: pgmap v254: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:19 smithi081 ceph-mon[35060]: pgmap v254: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:08:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:21 smithi102 ceph-mon[34458]: pgmap v255: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:21 smithi031 ceph-mon[31203]: pgmap v255: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:21 smithi081 ceph-mon[35060]: pgmap v255: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:23 smithi102 ceph-mon[34458]: pgmap v256: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:23 smithi031 ceph-mon[31203]: pgmap v256: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:23.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:23 smithi081 ceph-mon[35060]: pgmap v256: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:08:25.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:25 smithi081 ceph-mon[35060]: pgmap v257: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:08:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:25 smithi102 ceph-mon[34458]: pgmap v257: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:08:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:25 smithi031 ceph-mon[31203]: pgmap v257: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:08:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:27 smithi102 ceph-mon[34458]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:08:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:27 smithi031 ceph-mon[31203]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:08:27.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:27 smithi081 ceph-mon[35060]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:08:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:29 smithi102 ceph-mon[34458]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:29.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:29 smithi031 ceph-mon[31203]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:29.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:29 smithi081 ceph-mon[35060]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:31 smithi102 ceph-mon[34458]: pgmap v260: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:31 smithi031 ceph-mon[31203]: pgmap v260: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:31.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:31 smithi081 ceph-mon[35060]: pgmap v260: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:33 smithi102 ceph-mon[34458]: pgmap v261: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:33.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:33 smithi031 ceph-mon[31203]: pgmap v261: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:33 smithi081 ceph-mon[35060]: pgmap v261: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:35 smithi102 ceph-mon[34458]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:35 smithi031 ceph-mon[31203]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:35.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:35 smithi081 ceph-mon[35060]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:37.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:37 smithi102 ceph-mon[34458]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:37 smithi031 ceph-mon[31203]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:37.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:37 smithi081 ceph-mon[35060]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:39 smithi102 ceph-mon[34458]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:39 smithi031 ceph-mon[31203]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:39 smithi081 ceph-mon[35060]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:41 smithi102 ceph-mon[34458]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:41 smithi031 ceph-mon[31203]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:41 smithi081 ceph-mon[35060]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:43.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:43 smithi102 ceph-mon[34458]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:43.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:43 smithi031 ceph-mon[31203]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:43.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:43 smithi081 ceph-mon[35060]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:45.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:45 smithi102 ceph-mon[34458]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:45 smithi031 ceph-mon[31203]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:45.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:45 smithi081 ceph-mon[35060]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:47 smithi031 ceph-mon[31203]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:47.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:47 smithi081 ceph-mon[35060]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:47 smithi102 ceph-mon[34458]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:49.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:49 smithi031 ceph-mon[31203]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:49.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:49 smithi081 ceph-mon[35060]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:49 smithi102 ceph-mon[34458]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:51.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:51 smithi031 ceph-mon[31203]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:51.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:51 smithi081 ceph-mon[35060]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:51.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:51 smithi102 ceph-mon[34458]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:53 smithi031 ceph-mon[31203]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:53 smithi081 ceph-mon[35060]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:53 smithi102 ceph-mon[34458]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:55.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:55 smithi031 ceph-mon[31203]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:55.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:55 smithi081 ceph-mon[35060]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:55 smithi102 ceph-mon[34458]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:08:57.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:57 smithi031 ceph-mon[31203]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:57.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:57 smithi081 ceph-mon[35060]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:57 smithi102 ceph-mon[34458]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:59.423 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:08:59 smithi102 ceph-mon[34458]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:59.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:08:59 smithi031 ceph-mon[31203]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:08:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:08:59 smithi081 ceph-mon[35060]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:01 smithi031 ceph-mon[31203]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:01 smithi081 ceph-mon[35060]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:01 smithi102 ceph-mon[34458]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:03 smithi031 ceph-mon[31203]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:03.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:03 smithi081 ceph-mon[35060]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:03 smithi102 ceph-mon[34458]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:05.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:05 smithi031 ceph-mon[31203]: pgmap v277: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:09:05.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:05 smithi081 ceph-mon[35060]: pgmap v277: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:09:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:05 smithi102 ceph-mon[34458]: pgmap v277: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:09:07.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:07 smithi031 ceph-mon[31203]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:09:07.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:07 smithi081 ceph-mon[35060]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:09:07.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:07 smithi102 ceph-mon[34458]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:09:08.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:08 smithi031 ceph-mon[31203]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:09:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:08 smithi081 ceph-mon[35060]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:09:08.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:08 smithi102 ceph-mon[34458]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:09:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:09 smithi031 ceph-mon[31203]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:09.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:09 smithi081 ceph-mon[35060]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:09.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:09 smithi102 ceph-mon[34458]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:11 smithi031 ceph-mon[31203]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:11 smithi081 ceph-mon[35060]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:11 smithi102 ceph-mon[34458]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:13.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:13 smithi031 ceph-mon[31203]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:13 smithi081 ceph-mon[35060]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:13 smithi102 ceph-mon[34458]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:09:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:15 smithi102 ceph-mon[34458]: pgmap v283: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:15.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:15 smithi102 ceph-mon[34458]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:09:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:15 smithi031 ceph-mon[31203]: pgmap v283: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:15 smithi031 ceph-mon[31203]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:09:15.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:15 smithi081 ceph-mon[35060]: pgmap v283: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:15.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:15 smithi081 ceph-mon[35060]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:09:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:17 smithi102 ceph-mon[34458]: pgmap v285: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:09:17.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:17 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:17.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:17 smithi031 ceph-mon[31203]: pgmap v285: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:09:17.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:17 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:17 smithi081 ceph-mon[35060]: pgmap v285: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 241 B/s wr, 12 op/s 2024-09-16T00:09:17.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:17 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:18.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:18.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:18.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:09:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:09:18.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:18.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:09:19.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:19 smithi102 ceph-mon[34458]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:19.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:19 smithi031 ceph-mon[31203]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:19 smithi081 ceph-mon[35060]: pgmap v286: 97 pgs: 97 active+clean; 1.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:09:21.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:21 smithi102 ceph-mon[34458]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:21 smithi031 ceph-mon[31203]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:21 smithi081 ceph-mon[35060]: pgmap v287: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:23.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:23 smithi102 ceph-mon[34458]: pgmap v288: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:23.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:23 smithi031 ceph-mon[31203]: pgmap v288: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:23.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:23 smithi081 ceph-mon[35060]: pgmap v288: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-16T00:09:25.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:25 smithi081 ceph-mon[35060]: pgmap v289: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:09:25.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:25 smithi102 ceph-mon[34458]: pgmap v289: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:09:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:25 smithi031 ceph-mon[31203]: pgmap v289: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:09:27.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:27 smithi102 ceph-mon[34458]: pgmap v290: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:09:27.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:27 smithi031 ceph-mon[31203]: pgmap v290: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:09:27.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:27 smithi081 ceph-mon[35060]: pgmap v290: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:09:29.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:29 smithi102 ceph-mon[34458]: pgmap v291: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:29.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:29 smithi031 ceph-mon[31203]: pgmap v291: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:29.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:29 smithi081 ceph-mon[35060]: pgmap v291: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:31 smithi102 ceph-mon[34458]: pgmap v292: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:31 smithi031 ceph-mon[31203]: pgmap v292: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:31.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:31 smithi081 ceph-mon[35060]: pgmap v292: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:33 smithi102 ceph-mon[34458]: pgmap v293: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:33.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:33 smithi031 ceph-mon[31203]: pgmap v293: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:33.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:33 smithi081 ceph-mon[35060]: pgmap v293: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:35 smithi102 ceph-mon[34458]: pgmap v294: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:35 smithi031 ceph-mon[31203]: pgmap v294: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:35 smithi081 ceph-mon[35060]: pgmap v294: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:37 smithi031 ceph-mon[31203]: pgmap v295: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:37.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:37 smithi081 ceph-mon[35060]: pgmap v295: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:37.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:37 smithi102 ceph-mon[34458]: pgmap v295: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:39 smithi031 ceph-mon[31203]: pgmap v296: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:39 smithi081 ceph-mon[35060]: pgmap v296: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:39.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:39 smithi102 ceph-mon[34458]: pgmap v296: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:41 smithi031 ceph-mon[31203]: pgmap v297: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:41 smithi081 ceph-mon[35060]: pgmap v297: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:41.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:41 smithi102 ceph-mon[34458]: pgmap v297: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:43.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:43 smithi031 ceph-mon[31203]: pgmap v298: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:43.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:43 smithi081 ceph-mon[35060]: pgmap v298: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:43 smithi102 ceph-mon[34458]: pgmap v298: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:45 smithi031 ceph-mon[31203]: pgmap v299: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:45.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:45 smithi081 ceph-mon[35060]: pgmap v299: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:45.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:45 smithi102 ceph-mon[34458]: pgmap v299: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:47 smithi031 ceph-mon[31203]: pgmap v300: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:47.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:47 smithi081 ceph-mon[35060]: pgmap v300: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:47 smithi102 ceph-mon[34458]: pgmap v300: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:49.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:49 smithi031 ceph-mon[31203]: pgmap v301: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:49.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:49 smithi081 ceph-mon[35060]: pgmap v301: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:49 smithi102 ceph-mon[34458]: pgmap v301: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:51.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:51 smithi031 ceph-mon[31203]: pgmap v302: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:51.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:51 smithi081 ceph-mon[35060]: pgmap v302: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:51.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:51 smithi102 ceph-mon[34458]: pgmap v302: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:53 smithi031 ceph-mon[31203]: pgmap v303: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:53.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:53 smithi081 ceph-mon[35060]: pgmap v303: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:53.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:53 smithi102 ceph-mon[34458]: pgmap v303: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:55.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:55 smithi031 ceph-mon[31203]: pgmap v304: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:55.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:55 smithi081 ceph-mon[35060]: pgmap v304: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:55 smithi102 ceph-mon[34458]: pgmap v304: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:09:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:57 smithi031 ceph-mon[31203]: pgmap v305: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:57.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:57 smithi081 ceph-mon[35060]: pgmap v305: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:57 smithi102 ceph-mon[34458]: pgmap v305: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:59.423 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:09:59 smithi102 ceph-mon[34458]: pgmap v306: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:09:59 smithi031 ceph-mon[31203]: pgmap v306: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:09:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:09:59 smithi081 ceph-mon[35060]: pgmap v306: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:00.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:00 smithi031 ceph-mon[31203]: overall HEALTH_OK 2024-09-16T00:10:00.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:00 smithi081 ceph-mon[35060]: overall HEALTH_OK 2024-09-16T00:10:00.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:00 smithi102 ceph-mon[34458]: overall HEALTH_OK 2024-09-16T00:10:01.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:01 smithi031 ceph-mon[31203]: pgmap v307: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:01 smithi081 ceph-mon[35060]: pgmap v307: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:01.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:01 smithi102 ceph-mon[34458]: pgmap v307: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:03.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:03 smithi031 ceph-mon[31203]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:03.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:03 smithi081 ceph-mon[35060]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:03.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:03 smithi102 ceph-mon[34458]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:05.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:05 smithi031 ceph-mon[31203]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:10:05.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:05 smithi081 ceph-mon[35060]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:10:05.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:05 smithi102 ceph-mon[34458]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-16T00:10:07.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:07 smithi031 ceph-mon[31203]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:10:07.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:07 smithi081 ceph-mon[35060]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:10:07.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:07 smithi102 ceph-mon[34458]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-16T00:10:08.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:08 smithi031 ceph-mon[31203]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:10:08.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:08 smithi081 ceph-mon[35060]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:10:08.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:08 smithi102 ceph-mon[34458]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:10:09.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:09 smithi031 ceph-mon[31203]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:09.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:09 smithi081 ceph-mon[35060]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:09.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:09 smithi102 ceph-mon[34458]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:11.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:11 smithi031 ceph-mon[31203]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:11.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:11 smithi081 ceph-mon[35060]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:11.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:11 smithi102 ceph-mon[34458]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:11.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/385444864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb0101280 msgr2=0x7f6fb010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/385444864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb0101280 0x7f6fb010deb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009900 tx=0x7f6fa002a410 comp rx=0 tx=0).stop 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/385444864 shutdown_connections 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/385444864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb010e480 0x7f6fb0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/385444864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb0101280 0x7f6fb010deb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/385444864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 0x7f6fb0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/385444864 >> 172.21.15.31:0/385444864 conn(0x7f6fb00fc6b0 msgr2=0x7f6fb00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/385444864 shutdown_connections 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.862+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/385444864 wait complete. 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 Processor -- start 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 -- start start 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 0x7f6fb019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 0x7f6fb019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0114000 con 0x7f6fb010e480 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0113e80 con 0x7f6fb01008d0 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb7cdd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0114180 con 0x7f6fb0101280 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb627a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb627a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56142/0 (socket says 172.21.15.31:56142) 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.863+0000 7f6fb627a640 1 -- 172.21.15.31:0/1826849028 learned_addr learned my addr 172.21.15.31:0/1826849028 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb5a79640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 0x7f6fb019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb5278640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 0x7f6fb019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 -- 172.21.15.31:0/1826849028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 msgr2=0x7f6fb019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 0x7f6fb019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 -- 172.21.15.31:0/1826849028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 msgr2=0x7f6fb019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 0x7f6fb019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 -- 172.21.15.31:0/1826849028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb019f4e0 con 0x7f6fb010e480 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb5278640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 0x7f6fb019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb627a640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800e9d0 tx=0x7f6fa800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800cd80 con 0x7f6fb010e480 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fa8005020 con 0x7f6fb010e480 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb019f7d0 con 0x7f6fb010e480 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80116d0 con 0x7f6fb010e480 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.864+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb01a7060 con 0x7f6fb010e480 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.865+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb0100cb0 con 0x7f6fb010e480 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.867+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6fa80051d0 con 0x7f6fb010e480 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.867+0000 7f6fa6ffd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 0x7f6f9c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.868+0000 7f6fb5a79640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 0x7f6f9c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.868+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7f6fa80d0100 con 0x7f6fb010e480 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.868+0000 7f6fb5a79640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 0x7f6f9c0a5920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009bd0 tx=0x7f6fac005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:11.872+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fa8016030 con 0x7f6fb010e480 2024-09-16T00:10:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.070+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6fb0062630 con 0x7f6f9c0a3460 2024-09-16T00:10:12.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.078+0000 7f6fa6ffd640 1 -- 172.21.15.31:0/1826849028 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f6fb0062630 con 0x7f6f9c0a3460 2024-09-16T00:10:12.079 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-16T00:00:00", "created": "2024-09-16T00:00:09", "first": "2024-09-16T00:01:00", "last": "2024-09-16T00:10:00", "last_pruned": "2024-09-16T00:10:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-16T00:10:12.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 msgr2=0x7f6f9c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 0x7f6f9c0a5920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009bd0 tx=0x7f6fac005f90 comp rx=0 tx=0).stop 2024-09-16T00:10:12.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 msgr2=0x7f6fb019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800e9d0 tx=0x7f6fa800ee90 comp rx=0 tx=0).stop 2024-09-16T00:10:12.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 shutdown_connections 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f6f9c0a3460 0x7f6f9c0a5920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fb010e480 0x7f6fb019edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fb0101280 0x7f6fb019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 --2- 172.21.15.31:0/1826849028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fb01008d0 0x7f6fb019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 >> 172.21.15.31:0/1826849028 conn(0x7f6fb00fc6b0 msgr2=0x7f6fb00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 shutdown_connections 2024-09-16T00:10:12.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.080+0000 7f6fb7cdd640 1 -- 172.21.15.31:0/1826849028 wait complete. 2024-09-16T00:10:12.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- 172.21.15.31:0/3599958899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48102470 msgr2=0x7f9c4810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 --2- 172.21.15.31:0/3599958899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48102470 0x7f9c4810e9a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009cd0 tx=0x7f9c4002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- 172.21.15.31:0/3599958899 shutdown_connections 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 --2- 172.21.15.31:0/3599958899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48102470 0x7f9c4810e9a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 --2- 172.21.15.31:0/3599958899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48101ad0 0x7f9c48101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 --2- 172.21.15.31:0/3599958899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48107b70 0x7f9c48107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- 172.21.15.31:0/3599958899 >> 172.21.15.31:0/3599958899 conn(0x7f9c480fd7d0 msgr2=0x7f9c480ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- 172.21.15.31:0/3599958899 shutdown_connections 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- 172.21.15.31:0/3599958899 wait complete. 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 Processor -- start 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.198+0000 7f9c50465640 1 -- start start 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48102470 0x7f9c481987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48107b70 0x7f9c4819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48111dd0 con 0x7f9c48101ad0 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48111c50 con 0x7f9c48102470 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c50465640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48111f50 con 0x7f9c48107b70 2024-09-16T00:10:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56148/0 (socket says 172.21.15.31:56148) 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 -- 172.21.15.31:0/979496369 learned_addr learned my addr 172.21.15.31:0/979496369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4ea02640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48107b70 0x7f9c4819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4da00640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48102470 0x7f9c481987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 -- 172.21.15.31:0/979496369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48107b70 msgr2=0x7f9c4819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48107b70 0x7f9c4819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 -- 172.21.15.31:0/979496369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48102470 msgr2=0x7f9c481987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48102470 0x7f9c481987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 -- 172.21.15.31:0/979496369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4819d250 con 0x7f9c48101ad0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c4e201640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e9f0 tx=0x7f9c4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.199+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400cd80 con 0x7f9c48101ad0 2024-09-16T00:10:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.200+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c440108e0 con 0x7f9c48101ad0 2024-09-16T00:10:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.200+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4819d540 con 0x7f9c48101ad0 2024-09-16T00:10:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.200+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44018920 con 0x7f9c48101ad0 2024-09-16T00:10:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.200+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c481a4dd0 con 0x7f9c48101ad0 2024-09-16T00:10:12.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.201+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c48101f30 con 0x7f9c48101ad0 2024-09-16T00:10:12.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.202+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9c44010a90 con 0x7f9c48101ad0 2024-09-16T00:10:12.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.203+0000 7f9c3f7fe640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 0x7f9c340a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.203+0000 7f9c4da00640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 0x7f9c340a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.203+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7f9c440d03a0 con 0x7f9c48101ad0 2024-09-16T00:10:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.203+0000 7f9c4da00640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 0x7f9c340a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c480ffdc0 tx=0x7f9c38005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.206+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c4409c740 con 0x7f9c48101ad0 2024-09-16T00:10:12.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.403+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 --> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9c480fe630 con 0x7f9c340a34e0 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.410+0000 7f9c3f7fe640 1 -- 172.21.15.31:0/979496369 <== mgr.15216 v2:172.21.15.31:6800/225135964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9c480fe630 con 0x7f9c340a34e0 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 msgr2=0x7f9c340a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 0x7f9c340a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9c480ffdc0 tx=0x7f9c38005f90 comp rx=0 tx=0).stop 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 msgr2=0x7f9c48198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e9f0 tx=0x7f9c4400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 shutdown_connections 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f9c340a34e0 0x7f9c340a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c48107b70 0x7f9c4819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c48102470 0x7f9c481987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 --2- 172.21.15.31:0/979496369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c48101ad0 0x7f9c48198260 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 >> 172.21.15.31:0/979496369 conn(0x7f9c480fd7d0 msgr2=0x7f9c480fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 shutdown_connections 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.412+0000 7f9c50465640 1 -- 172.21.15.31:0/979496369 wait complete. 2024-09-16T00:10:12.424 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T00:10:12.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:10:12.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 -- 172.21.15.31:0/3481141870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 msgr2=0x7fc19c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 --2- 172.21.15.31:0/3481141870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c106db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009930 tx=0x7fc18c02a420 comp rx=0 tx=0).stop 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 -- 172.21.15.31:0/3481141870 shutdown_connections 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 --2- 172.21.15.31:0/3481141870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 0x7fc19c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 --2- 172.21.15.31:0/3481141870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 0x7fc19c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 --2- 172.21.15.31:0/3481141870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c106db0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 -- 172.21.15.31:0/3481141870 >> 172.21.15.31:0/3481141870 conn(0x7fc19c0fc6a0 msgr2=0x7fc19c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 -- 172.21.15.31:0/3481141870 shutdown_connections 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.541+0000 7fc1a125c640 1 -- 172.21.15.31:0/3481141870 wait complete. 2024-09-16T00:10:12.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.542+0000 7fc1a125c640 1 Processor -- start 2024-09-16T00:10:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.542+0000 7fc1a125c640 1 -- start start 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.542+0000 7fc1a125c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 0x7fc19c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.542+0000 7fc1a125c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 0x7fc19c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc1a125c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc1a125c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c110d10 con 0x7fc19c1069d0 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc1a125c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c110b90 con 0x7fc19c1009c0 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc1a125c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c110e90 con 0x7fc19c101360 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56172/0 (socket says 172.21.15.31:56172) 2024-09-16T00:10:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 -- 172.21.15.31:0/2048464562 learned_addr learned my addr 172.21.15.31:0/2048464562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19a59c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 0x7fc19c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19ad9d640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 0x7fc19c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 -- 172.21.15.31:0/2048464562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 msgr2=0x7fc19c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 0x7fc19c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 -- 172.21.15.31:0/2048464562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 msgr2=0x7fc19c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 0x7fc19c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 -- 172.21.15.31:0/2048464562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19d3b0 con 0x7fc19c1069d0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc19b59e640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc194009fd0 tx=0x7fc19400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194017070 con 0x7fc19c1069d0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19d6a0 con 0x7fc19c1069d0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc194003e20 con 0x7fc19c1069d0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.543+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194010460 con 0x7fc19c1069d0 2024-09-16T00:10:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.544+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a4f30 con 0x7fc19c1069d0 2024-09-16T00:10:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.545+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c100e20 con 0x7fc19c1069d0 2024-09-16T00:10:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.546+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc194003970 con 0x7fc19c1069d0 2024-09-16T00:10:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.546+0000 7fc183fff640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 0x7fc1840a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.546+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7fc1940cfa60 con 0x7fc19c1069d0 2024-09-16T00:10:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.546+0000 7fc19ad9d640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 0x7fc1840a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.547+0000 7fc19ad9d640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 0x7fc1840a5880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc18c031440 tx=0x7fc18c005a20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.550+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc19409be00 con 0x7fc19c1069d0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.727+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19c1a5190 con 0x7fc19c1069d0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.729+0000 7fc183fff640 1 -- 172.21.15.31:0/2048464562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+23106 (secure 0 0 0) 0x7fc1940a0100 con 0x7fc19c1069d0 2024-09-16T00:10:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:10:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-16T00:10:07.179742+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/540718575":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6801/797003742":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6800/797003742":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2677380564":"2024-09-17T00:00:07.493533+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/1147118595":"2024-09-17T00:00:07.493533+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3296404867":"2024-09-17T00:00:07.493533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:10:12.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 msgr2=0x7fc1840a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 0x7fc1840a5880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc18c031440 tx=0x7fc18c005a20 comp rx=0 tx=0).stop 2024-09-16T00:10:12.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 msgr2=0x7fc19c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc194009fd0 tx=0x7fc19400b260 comp rx=0 tx=0).stop 2024-09-16T00:10:12.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 shutdown_connections 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc1840a33c0 0x7fc1840a5880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc19c1069d0 0x7fc19c19cc80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc19c101360 0x7fc19c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 --2- 172.21.15.31:0/2048464562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c1009c0 0x7fc19c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 >> 172.21.15.31:0/2048464562 conn(0x7fc19c0fc6a0 msgr2=0x7fc19c06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 shutdown_connections 2024-09-16T00:10:12.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.731+0000 7fc1a125c640 1 -- 172.21.15.31:0/2048464562 wait complete. 2024-09-16T00:10:12.744 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T23:53:27.616831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T23:58:07.909795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T23:58:10.116305+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 90, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T00:10:12.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:10:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- 172.21.15.31:0/3718414248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc108130 msgr2=0x7f3bcc10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/3718414248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc108130 0x7f3bcc10ea80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009cd0 tx=0x7f3bc402a970 comp rx=0 tx=0).stop 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- 172.21.15.31:0/3718414248 shutdown_connections 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/3718414248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc108130 0x7f3bcc10ea80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/3718414248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc101200 0x7f3bcc1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/3718414248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- 172.21.15.31:0/3718414248 >> 172.21.15.31:0/3718414248 conn(0x7f3bcc075ea0 msgr2=0x7f3bcc0762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- 172.21.15.31:0/3718414248 shutdown_connections 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- 172.21.15.31:0/3718414248 wait complete. 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 Processor -- start 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.852+0000 7f3bd36c9640 1 -- start start 2024-09-16T00:10:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc101200 0x7f3bcc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc108130 0x7f3bcc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111e10 con 0x7f3bcc1008e0 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111c90 con 0x7f3bcc101200 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111f90 con 0x7f3bcc108130 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56202/0 (socket says 172.21.15.31:56202) 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 -- 172.21.15.31:0/1246748176 learned_addr learned my addr 172.21.15.31:0/1246748176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1c66640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc108130 0x7f3bcc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd0c64640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc101200 0x7f3bcc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 -- 172.21.15.31:0/1246748176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc108130 msgr2=0x7f3bcc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc108130 0x7f3bcc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 -- 172.21.15.31:0/1246748176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc101200 msgr2=0x7f3bcc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc101200 0x7f3bcc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 -- 172.21.15.31:0/1246748176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc19d3f0 con 0x7f3bcc1008e0 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd1465640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800d950 tx=0x7f3bc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8017070 con 0x7f3bcc1008e0 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bc8009c30 con 0x7f3bcc1008e0 2024-09-16T00:10:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800f620 con 0x7f3bcc1008e0 2024-09-16T00:10:12.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc19d6e0 con 0x7f3bcc1008e0 2024-09-16T00:10:12.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.853+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a4f70 con 0x7f3bcc1008e0 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.854+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc100cc0 con 0x7f3bcc1008e0 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.856+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3bc80109a0 con 0x7f3bcc1008e0 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.856+0000 7f3bc27fc640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 0x7f3bb80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.856+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7f3bc80cf7d0 con 0x7f3bcc1008e0 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.856+0000 7f3bd0c64640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 0x7f3bb80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.857+0000 7f3bd0c64640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 0x7f3bb80a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc100220 tx=0x7f3bbc00b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:12.860+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bc8009de0 con 0x7f3bcc1008e0 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.036+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bcc10a0c0 con 0x7f3bcc1008e0 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.037+0000 7f3bc27fc640 1 -- 172.21.15.31:0/1246748176 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3bc809bbf0 con 0x7f3bcc1008e0 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.038+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 msgr2=0x7f3bb80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 0x7f3bb80a5900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc100220 tx=0x7f3bbc00b390 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 msgr2=0x7f3bcc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800d950 tx=0x7f3bc800de10 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 shutdown_connections 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f3bb80a3440 0x7f3bb80a5900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3bcc108130 0x7f3bcc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3bcc101200 0x7f3bcc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 --2- 172.21.15.31:0/1246748176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bcc1008e0 0x7f3bcc198400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 >> 172.21.15.31:0/1246748176 conn(0x7f3bcc075ea0 msgr2=0x7f3bcc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 shutdown_connections 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.039+0000 7f3bd36c9640 1 -- 172.21.15.31:0/1246748176 wait complete. 2024-09-16T00:10:13.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.156+0000 7fde2f96d640 1 -- 172.21.15.31:0/2615234530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 msgr2=0x7fde28100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2615234530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde28100df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde18009930 tx=0x7fde1802a420 comp rx=0 tx=0).stop 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- 172.21.15.31:0/2615234530 shutdown_connections 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2615234530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 0x7fde2810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2615234530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde28100df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2615234530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde28106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- 172.21.15.31:0/2615234530 >> 172.21.15.31:0/2615234530 conn(0x7fde280fc690 msgr2=0x7fde280feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- 172.21.15.31:0/2615234530 shutdown_connections 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- 172.21.15.31:0/2615234530 wait complete. 2024-09-16T00:10:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 Processor -- start 2024-09-16T00:10:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- start start 2024-09-16T00:10:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 0x7fde280fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde280fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde28110c60 con 0x7fde28100990 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fde28110ae0 con 0x7fde28101330 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.157+0000 7fde2f96d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde28110de0 con 0x7fde281069a0 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56220/0 (socket says 172.21.15.31:56220) 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 -- 172.21.15.31:0/2935586614 learned_addr learned my addr 172.21.15.31:0/2935586614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2df0a640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde280fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2cf08640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 0x7fde280fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 -- 172.21.15.31:0/2935586614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 msgr2=0x7fde280fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde280fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 -- 172.21.15.31:0/2935586614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 msgr2=0x7fde280fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 0x7fde280fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 -- 172.21.15.31:0/2935586614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde281a6810 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2df0a640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde280fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2d709640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c920 tx=0x7fde2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24007c70 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde24007e10 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281a6b00 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24018820 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.158+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281a7020 con 0x7fde28100990 2024-09-16T00:10:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.159+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde28100df0 con 0x7fde28100990 2024-09-16T00:10:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.160+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fde240189c0 con 0x7fde28100990 2024-09-16T00:10:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.161+0000 7fde1e7fc640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 0x7fde140a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.161+0000 7fde2cf08640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 0x7fde140a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.162+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7fde240cfed0 con 0x7fde28100990 2024-09-16T00:10:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.162+0000 7fde2cf08640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 0x7fde140a5880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde18000c00 tx=0x7fde18032c80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.166+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde24014030 con 0x7fde28100990 2024-09-16T00:10:13.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.347+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde28109f30 con 0x7fde28100990 2024-09-16T00:10:13.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.347+0000 7fde1e7fc640 1 -- 172.21.15.31:0/2935586614 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fde2400de20 con 0x7fde28100990 2024-09-16T00:10:13.348 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 msgr2=0x7fde140a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 0x7fde140a5880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde18000c00 tx=0x7fde18032c80 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 msgr2=0x7fde280fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde2400c920 tx=0x7fde2400cde0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 shutdown_connections 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fde140a33c0 0x7fde140a5880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde281069a0 0x7fde280fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde28101330 0x7fde280fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 --2- 172.21.15.31:0/2935586614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde28100990 0x7fde280fce00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 >> 172.21.15.31:0/2935586614 conn(0x7fde280fc690 msgr2=0x7fde2806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 shutdown_connections 2024-09-16T00:10:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.350+0000 7fde2f96d640 1 -- 172.21.15.31:0/2935586614 wait complete. 2024-09-16T00:10:13.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:13 smithi031 ceph-mon[31203]: pgmap v314: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:13.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:13 smithi031 ceph-mon[31203]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:13 smithi031 ceph-mon[31203]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2048464562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:13.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:13 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1246748176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:13.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:10:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.467+0000 7efc90191640 1 -- 172.21.15.31:0/1186678605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88102420 msgr2=0x7efc8810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.467+0000 7efc90191640 1 --2- 172.21.15.31:0/1186678605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88102420 0x7efc8810e950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc80009cd0 tx=0x7efc8002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- 172.21.15.31:0/1186678605 shutdown_connections 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- 172.21.15.31:0/1186678605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88102420 0x7efc8810e950 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- 172.21.15.31:0/1186678605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88101a80 0x7efc88101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- 172.21.15.31:0/1186678605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 0x7efc88107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- 172.21.15.31:0/1186678605 >> 172.21.15.31:0/1186678605 conn(0x7efc880fd7c0 msgr2=0x7efc880ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- 172.21.15.31:0/1186678605 shutdown_connections 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- 172.21.15.31:0/1186678605 wait complete. 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 Processor -- start 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- start start 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88102420 0x7efc881987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 0x7efc8819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc88111d80 con 0x7efc88101a80 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efc88111c00 con 0x7efc88107a90 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.468+0000 7efc90191640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc88111f00 con 0x7efc88102420 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56246/0 (socket says 172.21.15.31:56246) 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 -- 172.21.15.31:0/1970041907 learned_addr learned my addr 172.21.15.31:0/1970041907 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8d72c640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88102420 0x7efc881987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8e72e640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 0x7efc8819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 -- 172.21.15.31:0/1970041907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88102420 msgr2=0x7efc881987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88102420 0x7efc881987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 -- 172.21.15.31:0/1970041907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 msgr2=0x7efc8819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 0x7efc8819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 -- 172.21.15.31:0/1970041907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8819d2a0 con 0x7efc88101a80 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc8df2d640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400cd80 con 0x7efc88101a80 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc840108e0 con 0x7efc88101a80 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8819d590 con 0x7efc88101a80 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84018920 con 0x7efc88101a80 2024-09-16T00:10:13.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.469+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc881a4e20 con 0x7efc88101a80 2024-09-16T00:10:13.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.470+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc88101ee0 con 0x7efc88101a80 2024-09-16T00:10:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.472+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efc84010a90 con 0x7efc88101a80 2024-09-16T00:10:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.473+0000 7efc7effd640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 0x7efc740a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.473+0000 7efc8d72c640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 0x7efc740a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.473+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7efc840d0390 con 0x7efc88101a80 2024-09-16T00:10:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.473+0000 7efc8d72c640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 0x7efc740a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc880ffe30 tx=0x7efc7800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.476+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc8409c730 con 0x7efc88101a80 2024-09-16T00:10:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:13 smithi081 ceph-mon[35060]: pgmap v314: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:13 smithi081 ceph-mon[35060]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:13 smithi081 ceph-mon[35060]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2048464562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:13.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:13 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1246748176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:13 smithi102 ceph-mon[34458]: pgmap v314: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-16T00:10:13.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:13 smithi102 ceph-mon[34458]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:13 smithi102 ceph-mon[34458]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2048464562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:13.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:13 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1246748176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:13.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.653+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc88199600 con 0x7efc88101a80 2024-09-16T00:10:13.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.653+0000 7efc7effd640 1 -- 172.21.15.31:0/1970041907 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7efc840a0a30 con 0x7efc88101a80 2024-09-16T00:10:13.655 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-16T00:10:13.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 msgr2=0x7efc740a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 0x7efc740a5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc880ffe30 tx=0x7efc7800b410 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 msgr2=0x7efc881982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 shutdown_connections 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7efc740a3440 0x7efc740a5900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc88107a90 0x7efc8819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc88102420 0x7efc881987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 --2- 172.21.15.31:0/1970041907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc88101a80 0x7efc881982b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 >> 172.21.15.31:0/1970041907 conn(0x7efc880fd7c0 msgr2=0x7efc880fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 shutdown_connections 2024-09-16T00:10:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.656+0000 7efc90191640 1 -- 172.21.15.31:0/1970041907 wait complete. 2024-09-16T00:10:13.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 -- 172.21.15.31:0/3349862404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 msgr2=0x7f1ff410e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 --2- 172.21.15.31:0/3349862404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff410e530 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009900 tx=0x7f1fec02a480 comp rx=0 tx=0).stop 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 -- 172.21.15.31:0/3349862404 shutdown_connections 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 --2- 172.21.15.31:0/3349862404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff410e530 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 --2- 172.21.15.31:0/3349862404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.772+0000 7f1ff9531640 1 --2- 172.21.15.31:0/3349862404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 0x7f1ff41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 -- 172.21.15.31:0/3349862404 >> 172.21.15.31:0/3349862404 conn(0x7f1ff4077a10 msgr2=0x7f1ff40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 -- 172.21.15.31:0/3349862404 shutdown_connections 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 -- 172.21.15.31:0/3349862404 wait complete. 2024-09-16T00:10:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 Processor -- start 2024-09-16T00:10:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 -- start start 2024-09-16T00:10:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 0x7f1ff4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.773+0000 7f1ff9531640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff9531640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4111d40 con 0x7f1ff410c0c0 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff9531640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4111bc0 con 0x7f1ff4103610 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff9531640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4111ec0 con 0x7f1ff4102cf0 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56254/0 (socket says 172.21.15.31:56254) 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 -- 172.21.15.31:0/1294890731 learned_addr learned my addr 172.21.15.31:0/1294890731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff2ffd640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 0x7f1ff4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff27fc640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 -- 172.21.15.31:0/1294890731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 msgr2=0x7f1ff4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 0x7f1ff4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 -- 172.21.15.31:0/1294890731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 msgr2=0x7f1ff4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff4198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 -- 172.21.15.31:0/1294890731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff419d320 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff27fc640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff37fe640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009230 tx=0x7f1fec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03f030 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fec003aa0 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff419d5b0 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec035070 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.774+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41a4ea0 con 0x7f1ff410c0c0 2024-09-16T00:10:13.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.775+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff41030d0 con 0x7f1ff410c0c0 2024-09-16T00:10:13.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.777+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1fec042030 con 0x7f1ff410c0c0 2024-09-16T00:10:13.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.778+0000 7f1fdbfff640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 0x7f1fdc0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.778+0000 7f1ff2ffd640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 0x7f1fdc0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.778+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8507+0+0 (secure 0 0 0) 0x7f1fec0f09b0 con 0x7f1ff410c0c0 2024-09-16T00:10:13.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.779+0000 7f1ff2ffd640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 0x7f1fdc0a5880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005fd0 tx=0x7f1fe4009870 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:13.781+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fec0bd3a0 con 0x7f1ff410c0c0 2024-09-16T00:10:14.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.008+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1ff4107170 con 0x7f1ff410c0c0 2024-09-16T00:10:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.016+0000 7f1fdbfff640 1 -- 172.21.15.31:0/1294890731 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742869 (secure 0 0 0) 0x7f1fec0bcca0 con 0x7f1ff410c0c0 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 26, 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 15216, 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-09-16T00:10:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 225135964 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 225135964 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/225135964", 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-09-16T00:00:07.493709+0000", 2024-09-16T00:10:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24880, 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:10:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:10:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:10:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:10:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:10:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:10:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:10:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:10:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:10:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:10:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:10:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:10:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:10:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:10:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:10:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:10:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:10:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:10:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:10:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:10:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:10:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:10:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:10:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:10:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:10:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:10:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:10:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:10:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:10:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:10:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:10:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:10:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:10:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:10:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:10:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:10:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:10:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:10:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:10:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:10:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:10:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:10:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:10:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:10:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:10:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:10:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:10:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:10:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:10:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:10:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:10:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:10:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:10:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:10:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:10:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:10:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:10:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:10:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:10:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:10:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:10:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:10:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:10:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:10:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24890, 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:10:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:10:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:10:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:10:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:10:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:10:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:10:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:10:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:10:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:10:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:10:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:10:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:10:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:10:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:10:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:10:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:10:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:10:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:10:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:10:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:10:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:10:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:10:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:10:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:10:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:10:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:10:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:10:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:10:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:10:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:10:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:10:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:10:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:10:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:10:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:10:14.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:10:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:10:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:10:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:10:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:10:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:10:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:10:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:10:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:10:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:10:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:10:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:10:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.881 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.882 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.885 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:10:14.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:10:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:10:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:10:14.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.910 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.913 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:10:14.916 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.917 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.918 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.920 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.921 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.927 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:10:14.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:10:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:10:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:10:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:10:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:10:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:10:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:10:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-09-16T00:10:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-16T00:10:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-16T00:10:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-16T00:10:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-16T00:10:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-16T00:10:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-16T00:10:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-16T00:10:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-16T00:10:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-16T00:10:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:10:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:15.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.051 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-16T00:10:15.052 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.055 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-16T00:10:15.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:10:15.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-16T00:10:15.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.068 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:10:15.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-16T00:10:15.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.075 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-16T00:10:15.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-16T00:10:15.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.081 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:10:15.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:15.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.088 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.093 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:10:15.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.097 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:10:15.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:10:15.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-16T00:10:15.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-16T00:10:15.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-16T00:10:15.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-16T00:10:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:10:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-16T00:10:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:10:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-16T00:10:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-16T00:10:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:10:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:10:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:10:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:10:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:10:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:10:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-16T00:10:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:10:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-16T00:10:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-16T00:10:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:10:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:10:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-16T00:10:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-16T00:10:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-16T00:10:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.283 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-16T00:10:15.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:10:15.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-16T00:10:15.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.293 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.296 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.311 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-16T00:10:15.313 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.314 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-16T00:10:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-16T00:10:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-16T00:10:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-16T00:10:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:10:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-16T00:10:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-16T00:10:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-16T00:10:15.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-16T00:10:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.437 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.441 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.445 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.463 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.467 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.470 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-16T00:10:15.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:10:15.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-16T00:10:15.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-16T00:10:15.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.493 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.494 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-16T00:10:15.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-16T00:10:15.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.503 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.504 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.505 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.505 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.505 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.505 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.505 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.513 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.514 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.515 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.517 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:10:15.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.523 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-16T00:10:15.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-16T00:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.529 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-16T00:10:15.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-16T00:10:15.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-16T00:10:15.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-09-16T00:10:15.540 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-09-16T00:10:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T00:10:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 81, 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1618897904 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3060597214 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 519037131 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2214822331 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3247516015 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-16T00:10:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1718345205 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T00:10:15.551 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:10:15.552 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2935586614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1970041907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:14 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1294890731' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:14 smithi102 ceph-mon[34458]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2935586614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1970041907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:14 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1294890731' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:10:15.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:14 smithi081 ceph-mon[35060]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:10:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.021+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 msgr2=0x7f1fdc0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.021+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 0x7f1fdc0a5880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005fd0 tx=0x7f1fe4009870 comp rx=0 tx=0).stop 2024-09-16T00:10:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.021+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 msgr2=0x7f1ff419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.021+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009230 tx=0x7f1fec004270 comp rx=0 tx=0).stop 2024-09-16T00:10:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 shutdown_connections 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f1fdc0a33c0 0x7f1fdc0a5880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff410c0c0 0x7f1ff419cbf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ff4103610 0x7f1ff4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 --2- 172.21.15.31:0/1294890731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4102cf0 0x7f1ff4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 >> 172.21.15.31:0/1294890731 conn(0x7f1ff4077a10 msgr2=0x7f1ff40fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 shutdown_connections 2024-09-16T00:10:15.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:14.022+0000 7f1ff9531640 1 -- 172.21.15.31:0/1294890731 wait complete. 2024-09-16T00:10:15.562 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T00:10:15.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T00:10:15.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2935586614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1970041907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:10:15.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:14 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1294890731' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:10:15.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:14 smithi031 ceph-mon[31203]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:10:15.563 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:15 smithi031 ceph-mon[31203]: pgmap v315: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:15.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:15 smithi081 ceph-mon[35060]: pgmap v315: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:15.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:15 smithi102 ceph-mon[34458]: pgmap v315: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:15.832 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1433, 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1433, 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 432136, 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 454871, 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-09-16T00:10:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.152904671, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.048388668, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.042346121, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.047157189, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1432, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.060835010, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000042482 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 615, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 765, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 151770, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 625210, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 7, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.081135149, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.026975717, 2024-09-16T00:10:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.019140996, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.026820894, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 762, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.033398961, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000043830 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 430208 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 229, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 216, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1619340, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 125230, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.040172831, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.015158139, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.036894807, 2024-09-16T00:10:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000019078, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 215, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.013551175, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000063028 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1619072, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 124912 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3219, 2024-09-16T00:10:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.103578730, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000032177 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2024-09-16T00:10:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-09-16T00:10:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-09-16T00:10:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-09-16T00:10:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2024-09-16T00:10:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 7, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-09-16T00:10:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4196455, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 4196314, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 7 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 3, 2024-09-16T00:10:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 3, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 48288, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 215328 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-09-16T00:10:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-09-16T00:10:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-09-16T00:10:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-09-16T00:10:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 1, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 3, 2024-09-16T00:10:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 2, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 3, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 3723, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 189473, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 53, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 8064, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 179, 2024-09-16T00:10:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 8512, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 13, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-09-16T00:10:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2043, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2043, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2043, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 1.478975109, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000723923 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2043, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2044 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2043, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2043, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1322, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 719, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2024-09-16T00:10:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-09-16T00:10:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 91, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 11, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 5, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 5, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-09-16T00:10:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-09-16T00:10:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2273, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1984038, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2273, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2273, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1984038, 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2043, 2024-09-16T00:10:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2156, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2043, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2156, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2043, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2043, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2043, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2043, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55c5557deb40": { 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-16T00:10:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:16.516 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:10:17.419 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:17 smithi031 ceph-mon[31203]: pgmap v317: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 241 B/s wr, 15 op/s 2024-09-16T00:10:17.450 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-16T00:10:17.450 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-16T00:10:17.450 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-16T00:10:17.450 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:17.450 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:17.450 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T00:10:17.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T00:10:17.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:17 smithi081 ceph-mon[35060]: pgmap v317: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 241 B/s wr, 15 op/s 2024-09-16T00:10:17.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:17 smithi102 ceph-mon[34458]: pgmap v317: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 241 B/s wr, 15 op/s 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:18.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001451351, 2024-09-16T00:10:18.365 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000251280, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:18.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 219, 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 206, 2024-09-16T00:10:18.367 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1587520, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 125792, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.043811536, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.015330986, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.040616088, 2024-09-16T00:10:18.368 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 206, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017230134, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000083641 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1587520, 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 125792 2024-09-16T00:10:18.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 156, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 184496, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014134630, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009232469, 2024-09-16T00:10:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002409062, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 153, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017366447, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000113506 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 184496 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.371 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:18.372 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:18.373 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148994, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 31, 2024-09-16T00:10:18.374 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:10:18.375 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 217, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1554250, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:18.376 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 217, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 217, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1554250, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:18.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:10:18.479 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:18 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:18.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:18 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:18.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:18 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:19.016 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T00:10:19.016 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T00:10:19.016 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-16T00:10:19.016 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-16T00:10:19.016 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-16T00:10:19.016 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T00:10:19.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T00:10:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:19 smithi031 ceph-mon[31203]: pgmap v318: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:19 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:19 smithi081 ceph-mon[35060]: pgmap v318: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:19 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:19 smithi102 ceph-mon[34458]: pgmap v318: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-16T00:10:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:19 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:10:20.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:20 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:10:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:10:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 721, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 721, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 243801, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 228455, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.087285131, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.035387880, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.020720219, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.021693743, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 720, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.045872005, 2024-09-16T00:10:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000063711 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1592, 2024-09-16T00:10:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1579, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1969266, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 559365, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.135906897, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.040153712, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.066656331, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.035640604, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1578, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.053901721, 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000034158 2024-09-16T00:10:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1619616, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 126192 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 5, 2024-09-16T00:10:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 157, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1668, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 434757, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 7, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.012749464, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008450439, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002303514, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000051915, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 154, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.016904213, 2024-09-16T00:10:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000109767 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 433792 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-09-16T00:10:20.424 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3507, 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.091714584, 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000026151 2024-09-16T00:10:20.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-09-16T00:10:20.426 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 0, 2024-09-16T00:10:20.427 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2024-09-16T00:10:20.428 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2024-09-16T00:10:20.429 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:20.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-09-16T00:10:20.431 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:20.432 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.433 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 10, 2024-09-16T00:10:20.434 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-09-16T00:10:20.435 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-09-16T00:10:20.435 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4197382, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 4197241, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.436 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 10 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 3, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 3, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 0, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 0, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 0, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 0, 2024-09-16T00:10:20.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 48388, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 215328 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-09-16T00:10:20.438 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.439 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.440 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.443 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.443 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-09-16T00:10:20.443 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.444 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.445 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.446 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:20.447 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.449 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 2, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 5, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 3, 2024-09-16T00:10:20.451 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 4, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 2685, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:20.452 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:20.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 197490, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 56, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:20.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 176, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:10:20.456 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 8512, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 13, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-09-16T00:10:20.457 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2082, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2082, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2082, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 1.080981822, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000519203 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-09-16T00:10:20.458 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2082, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2083 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2082, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2082, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1354, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2024-09-16T00:10:20.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 726, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2024-09-16T00:10:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-09-16T00:10:20.461 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-09-16T00:10:20.462 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 91, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 11, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 6, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 6, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2024-09-16T00:10:20.463 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:20.464 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2314, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1991733, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:20.465 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2314, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2314, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1991733, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:20.466 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2082, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 3083, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2082, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 3083, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.467 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:20.468 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2082, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2082, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2082, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2082, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-16T00:10:20.469 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55fc74824b40": { 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:10:20.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:10:20.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:20 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi081 to 176.0M 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi102 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-16T00:10:20.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:20 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi081 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-16T00:10:21.261 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-16T00:10:21.262 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-16T00:10:21.262 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-16T00:10:21.262 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:21.262 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:21.262 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T00:10:21.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T00:10:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:21 smithi031 ceph-mon[31203]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:21.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:21 smithi081 ceph-mon[35060]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:21.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:21 smithi102 ceph-mon[34458]: pgmap v319: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T00:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 221, 2024-09-16T00:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 208, 2024-09-16T00:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1588064, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 127072, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.045613823, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.014760612, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.043301481, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 208, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.016714622, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000080358 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1588064, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 127072 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 158, 2024-09-16T00:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 184848, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.013928329, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009485695, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001948955, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 155, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.012598834, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000081282 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 184848 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-16T00:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000312350, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000234724, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-16T00:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148977, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-16T00:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-16T00:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 219, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1554510, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 219, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 219, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1554510, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T00:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T00:10:22.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:22.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:22 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.717 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T00:10:22.717 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T00:10:22.717 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-16T00:10:22.717 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-16T00:10:22.717 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-16T00:10:22.717 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T00:10:22.718 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T00:10:22.870 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:22.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:22 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:22.978 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: pgmap v320: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:23.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:22 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 3298, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1999, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1242937, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 1310395, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 15, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.310987769, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.078339416, 2024-09-16T00:10:23.528 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.129048774, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.106535924, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2000, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.004726036, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000502363 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2614, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000511958, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000195 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 347200, 2024-09-16T00:10:23.529 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 115888 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 2474, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 2269, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2276114, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 806933, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 10, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.186018476, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.063106034, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.101733362, 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.039159067, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2272, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.895844811, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000834438 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 734, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000146432, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000199 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1653328, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 132848 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1800, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1526, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 780101, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 879347, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 14, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 9, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.149085530, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.053735646, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.086766175, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.023406912, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1517, 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.283511746, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000186889 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1589, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000300680, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000189 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 43472, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 442096 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:10:23.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 390, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.069857340, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000179121 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2852, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.078303236, 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000027455 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 157, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 157, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 157, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.143645372, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000914938 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 12, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 34, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 1, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 33, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 32, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 3, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 3, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 2, 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 195, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 185, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-09-16T00:10:23.537 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 234, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:23.538 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 17, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 72, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 82, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 58, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 5, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 31 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 9, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 2, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 5, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 3, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 184, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 184, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 2, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 2, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 4775704, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 153, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.285884942, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001868529 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 26, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 29, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 3, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 20, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 20, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 0, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 34, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 37, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 3, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 3, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 21, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 18, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 54680, 2024-09-16T00:10:23.544 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 231712 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 205, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 1, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 157, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 27, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 2, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.006675560, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000953651 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.545 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 97, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.017873993, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000184267 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-09-16T00:10:23.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000357202, 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000178601 2024-09-16T00:10:23.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.548 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 10, 2024-09-16T00:10:23.549 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001495005, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000149500 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.012208154, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001526019 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-09-16T00:10:23.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 10, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.050697916, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005069791 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001154813, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000577406 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.551 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001237380, 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000618690 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.002212299, 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001106149 2024-09-16T00:10:23.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:23.553 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.005473512, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002736756 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.018712620, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.004678155 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.554 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001427426, 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001427426 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:23.555 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.556 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.018248099, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002606871 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.557 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.005871393, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001957131 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 4, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 27, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 13, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 588, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:10:23.559 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:23.560 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:23.561 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 371729, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 148, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 440, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 5, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:10:23.562 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 176, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 129740, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 880, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:10:23.563 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 3830, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 599451, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 37, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 3830, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3830, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 29.437436423, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007686014 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3830, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3887 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 3830, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 3, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 3827, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 11, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 3, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 156, 2024-09-16T00:10:23.565 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 31, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 3584, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 22, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:23.566 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-09-16T00:10:23.567 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 91, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 11, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-09-16T00:10:23.568 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 64, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 0, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 16 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-09-16T00:10:23.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 2560, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 1, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 3072, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 3, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 7568, 2024-09-16T00:10:23.570 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 3559998, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 7568, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 7568, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 3559998, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:23.571 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 3830, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 599762, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3830, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 599762, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.572 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 3830, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 3830, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3830, 2024-09-16T00:10:23.573 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 3830, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 3, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 311, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:23.574 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 3, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 311, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55c05ee6e960": { 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:23.575 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 184, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 581400, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 184, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 153, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 581400, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:23.576 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:10:24.430 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-16T00:10:24.430 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-16T00:10:24.430 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-16T00:10:24.430 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-16T00:10:24.430 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-16T00:10:24.430 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T00:10:24.430 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T00:10:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:10:24.974 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:24 smithi081 ceph-mon[35060]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:10:25.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:24 smithi031 ceph-mon[31203]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:10:25.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:24 smithi102 ceph-mon[34458]: pgmap v321: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-16T00:10:25.250 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:10:25.251 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:25.251 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1751, 2024-09-16T00:10:25.251 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1738, 2024-09-16T00:10:25.251 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2652212, 2024-09-16T00:10:25.251 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 611004, 2024-09-16T00:10:25.252 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 5, 2024-09-16T00:10:25.252 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 1, 2024-09-16T00:10:25.252 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.175136247, 2024-09-16T00:10:25.252 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.067351826, 2024-09-16T00:10:25.252 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.073862314, 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.036863994, 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1737, 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.067186372, 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000038679 2024-09-16T00:10:25.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:25.254 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.254 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.254 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1620864, 2024-09-16T00:10:25.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 128912 2024-09-16T00:10:25.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.255 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:25.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 6, 2024-09-16T00:10:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 162, 2024-09-16T00:10:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1976, 2024-09-16T00:10:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 437450, 2024-09-16T00:10:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-09-16T00:10:25.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:25.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.010105278, 2024-09-16T00:10:25.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.006222374, 2024-09-16T00:10:25.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.002067504, 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000066217, 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 159, 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.016708018, 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000105081 2024-09-16T00:10:25.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:25.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 436112 2024-09-16T00:10:25.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.260 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:25.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 723, 2024-09-16T00:10:25.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 723, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 179924, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 229097, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.061890667, 2024-09-16T00:10:25.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.017684596, 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.016174606, 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.022882968, 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 722, 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.025022110, 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000034656 2024-09-16T00:10:25.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-16T00:10:25.264 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:10:25.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.264 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:10:25.264 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-09-16T00:10:25.264 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3744, 2024-09-16T00:10:25.265 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.089869266, 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000024003 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:25.266 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.267 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-09-16T00:10:25.268 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2024-09-16T00:10:25.268 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2024-09-16T00:10:25.268 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-09-16T00:10:25.268 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.268 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:25.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 26, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 0, 2024-09-16T00:10:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 26, 2024-09-16T00:10:25.271 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:25.271 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 26, 2024-09-16T00:10:25.271 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-09-16T00:10:25.271 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2024-09-16T00:10:25.271 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 2, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:25.272 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-09-16T00:10:25.273 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2024-09-16T00:10:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 234, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:25.275 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:25.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:25.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:25.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:25.277 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-09-16T00:10:25.277 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:25.277 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-09-16T00:10:25.278 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-09-16T00:10:25.279 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:25.279 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-09-16T00:10:25.279 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:25.279 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:25.279 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:25.280 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:25.281 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-09-16T00:10:25.282 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 184, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-09-16T00:10:25.283 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 2, 2024-09-16T00:10:25.284 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-09-16T00:10:25.285 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-09-16T00:10:25.285 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-09-16T00:10:25.285 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 4775704, 2024-09-16T00:10:25.285 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 4775592, 2024-09-16T00:10:25.285 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 184 2024-09-16T00:10:25.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 17, 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 20, 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 3, 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 11, 2024-09-16T00:10:25.287 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 11, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 0, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 26, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 26, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 0, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2024-09-16T00:10:25.288 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2024-09-16T00:10:25.289 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-09-16T00:10:25.289 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 48252, 2024-09-16T00:10:25.289 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 215328 2024-09-16T00:10:25.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.289 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-09-16T00:10:25.290 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-09-16T00:10:25.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.292 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.292 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:25.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.293 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 4, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 3, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 2164, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 189410, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 52, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 176, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 87476, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 476, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 2239, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 26, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 4, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 2239, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2239, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.857983525, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000383199 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2239, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2240 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 2239, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 2239, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 1437, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 799, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 2, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 91, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 11, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 5, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 5, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 2476, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 2596276, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 2476, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2476, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2596276, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2239, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 581410, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2239, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 581410, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2239, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 2239, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2239, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2239, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55aacd69eb40": { 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:10:26.118 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T00:10:26.118 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T00:10:26.118 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-16T00:10:26.118 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:26.118 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:26.118 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T00:10:26.118 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T00:10:26.365 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:10:26.929 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 347, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 355, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1669622, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 217564, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 14, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.054661118, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.017788469, 2024-09-16T00:10:26.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.050925815, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000298431, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 355, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.031011942, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000087357 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 130, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000029654, 2024-09-16T00:10:26.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000228 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1643616, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 133360 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 66, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 220, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 24913, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 468829, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 10, 2024-09-16T00:10:26.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.017850961, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.007643103, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.008334555, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000382543, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 217, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.031390855, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000144658 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 83, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000017830, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000214 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 435760 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 339, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 523, 2024-09-16T00:10:26.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 104323, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 209065, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 8, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 4, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.055445658, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.025906730, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.023118323, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.004074273, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 521, 2024-09-16T00:10:26.935 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.039497690, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000075811 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 823, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000241169, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000293 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:10:26.936 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 44, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001473730, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000033493 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.937 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000008924, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000004462 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 1, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 1, 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-09-16T00:10:26.938 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000440760, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000440760 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 11, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-09-16T00:10:26.939 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 11, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 0, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 11, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 11, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 1, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 1, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 2, 2024-09-16T00:10:26.940 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 2, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 1, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 1, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-09-16T00:10:26.941 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 234, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:26.942 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:26.943 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 2, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-09-16T00:10:26.944 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:26.945 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 3, 2024-09-16T00:10:26.946 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 3, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 2, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 2, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-09-16T00:10:26.947 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 4195387, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.003559017, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001779508 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-09-16T00:10:26.948 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 14, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 14, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 0, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 12, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 13, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 1, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 11, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 11, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 0, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 1, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 1, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-09-16T00:10:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 52676, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 223520 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 2, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 1, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 9, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000440760, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000440760 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.951 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.952 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-09-16T00:10:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.956 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.956 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.958 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.959 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.961 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-09-16T00:10:26.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:26.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 1, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 0, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-09-16T00:10:26.965 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 1, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 1, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 618, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:26.966 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:26.967 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:26.968 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:26.969 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 262278, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 86, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:26.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 176, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 55448, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 215, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:26.971 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 38, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 11389, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 33, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 38, 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-09-16T00:10:26.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 38, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.194996296, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005131481 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 38, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 61 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 38, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 0, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 38, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 10, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 0, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 2, 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 6, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 6, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 0, 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 20, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 91, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 11, 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 8, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 8, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 16, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 0, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 1 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-09-16T00:10:26.980 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.981 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 748, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 1715028, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 748, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 748, 2024-09-16T00:10:26.982 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 1715028, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:26.983 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 38, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 11389, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 38, 2024-09-16T00:10:26.984 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 11389, 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:26.985 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 38, 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 38, 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 38, 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 38, 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x558326d72960": { 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 3, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 1083, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 3, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 1083, 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:10:27.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:26 smithi031 ceph-mon[31203]: pgmap v322: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:10:27.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:26 smithi081 ceph-mon[35060]: pgmap v322: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:10:27.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:26 smithi102 ceph-mon[34458]: pgmap v322: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:10:27.754 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-16T00:10:27.754 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-16T00:10:27.754 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-16T00:10:27.755 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:27.755 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:27.755 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T00:10:27.755 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T00:10:28.000 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:10:28.541 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-16T00:10:28.541 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:28.541 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 729, 2024-09-16T00:10:28.541 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 731, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 192688, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 231886, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 6, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.069134834, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.028103662, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.016438286, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.017274561, 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:28.542 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 729, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.028579336, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000039203 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:10:28.543 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 630, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 786, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 156790, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 635349, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.069400250, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.024180747, 2024-09-16T00:10:28.544 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.015694597, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.018999792, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 784, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.045859902, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000058494 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.545 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 436912 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 962, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 949, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1865056, 2024-09-16T00:10:28.546 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 361544, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 3, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.126143663, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.041808675, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.065330294, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.026230536, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 948, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.055844108, 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000058907 2024-09-16T00:10:28.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1621792, 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 131312 2024-09-16T00:10:28.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3446, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.096127747, 2024-09-16T00:10:28.549 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000027895 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:28.551 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-09-16T00:10:28.552 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 1, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-09-16T00:10:28.553 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2024-09-16T00:10:28.554 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:28.555 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-09-16T00:10:28.556 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:28.557 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:28.558 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-09-16T00:10:28.559 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 3, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 2, 2024-09-16T00:10:28.560 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 4195387, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 4194892, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 3 2024-09-16T00:10:28.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 3, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 3, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 1, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 1, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2024-09-16T00:10:28.562 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 48448, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 215328 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2024-09-16T00:10:28.563 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.564 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.566 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.567 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.568 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-09-16T00:10:28.569 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.570 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.570 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.570 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.572 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.573 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.574 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.575 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.576 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.577 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.578 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:28.579 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2024-09-16T00:10:28.580 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 1, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 0, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 1652, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-09-16T00:10:28.581 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:28.582 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:28.583 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:28.584 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 189423, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 51, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:28.585 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 176, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:28.586 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 8512, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 13, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.587 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 2077, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 21, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 3, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 2077, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2077, 2024-09-16T00:10:28.588 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.960981523, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000462677 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2077, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2078 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 2077, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 2077, 2024-09-16T00:10:28.589 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 1346, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 729, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-09-16T00:10:28.590 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 1, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-09-16T00:10:28.591 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-09-16T00:10:28.592 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 91, 2024-09-16T00:10:28.593 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 11, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 5, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 5, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2024-09-16T00:10:28.594 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:28.595 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 2317, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 1990846, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 2317, 2024-09-16T00:10:28.596 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2317, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 1990846, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2077, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 1088, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2077, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 1088, 2024-09-16T00:10:28.598 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:28.599 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2077, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 2077, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2077, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2077, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.600 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:28.601 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55aa68e34b40": { 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-09-16T00:10:28.602 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-09-16T00:10:28.603 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-16T00:10:28.604 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-16T00:10:29.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:28 smithi031 ceph-mon[31203]: pgmap v323: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:29.056 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:28 smithi081 ceph-mon[35060]: pgmap v323: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:29.120 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:28 smithi102 ceph-mon[34458]: pgmap v323: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:29.356 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-16T00:10:29.357 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-16T00:10:29.357 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-16T00:10:29.357 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:29.357 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:29.357 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T00:10:29.357 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T00:10:29.620 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 688, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 833, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1772650, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 404260, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 10, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.099846853, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.037493094, 2024-09-16T00:10:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.067860475, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.004844787, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 836, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.059190300, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000070801 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 945, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000230018, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000243 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1635744, 2024-09-16T00:10:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 133552 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 74, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 104, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 28762, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 153894, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 11, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.012579736, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.003800759, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.008297895, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000232550, 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 102, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.027801469, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000272563 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 98, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000023892, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000243 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 4960, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 127712 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 30, 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 189, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 6650, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 382090, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 13, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.013008508, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006135831, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.005160800, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000303718, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 183, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.031170156, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000170328 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 48, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000012728, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000265 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 312240 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-16T00:10:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 48, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.001381510, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000028781 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000013137, 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000006568 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 2, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 1, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000720482, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000720482 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 1, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-09-16T00:10:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 14, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 14, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 14, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 1, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 1, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 2, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 3, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 1, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-16T00:10:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 1, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 1000, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 1, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-09-16T00:10:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 1, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 1, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 1 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 2, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-16T00:10:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 7, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 7, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 2, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-16T00:10:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 2, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 4196455, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 6, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010544558, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001757426 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-16T00:10:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 17, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 17, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 0, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 15, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 16, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 1, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 14, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 14, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 0, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 1, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 1, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 50724, 2024-09-16T00:10:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223520 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 3, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 2, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 17, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-16T00:10:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000720482, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000720482 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-16T00:10:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-16T00:10:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-16T00:10:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 3, 2024-09-16T00:10:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 2, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 1, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 1, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 620, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 255099, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 87, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 176, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 68037, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 288, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 43, 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 12722, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 43, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 43, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.225995408, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005255707 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 43, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 66 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 43, 2024-09-16T00:10:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 43, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 6, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 7, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 6, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-16T00:10:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-16T00:10:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-16T00:10:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 91, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 11, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 8, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 8, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 16, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 1 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-16T00:10:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 788, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1720336, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 788, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 788, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1720336, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 43, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 12722, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 43, 2024-09-16T00:10:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 12722, 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 43, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 43, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 43, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 43, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:10:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55ae58ce8960": { 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 7, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 2151, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 7, 2024-09-16T00:10:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 6, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2151, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:30.164 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:10:31.097 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T00:10:31.097 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-16T00:10:31.097 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-16T00:10:31.097 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:31.097 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:31.097 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T00:10:31.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T00:10:31.347 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:10:31.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:31 smithi102 ceph-mon[34458]: pgmap v324: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:31.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:31 smithi031 ceph-mon[31203]: pgmap v324: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:31.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:31 smithi081 ceph-mon[35060]: pgmap v324: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:31.872 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:10:31.872 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:31.872 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 396, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 508, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1678542, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 526250, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 15, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.064984153, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.024511001, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.053433317, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000361837, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 508, 2024-09-16T00:10:31.873 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.045105669, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000088790 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 201, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000042579, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000211 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1632800, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 450688 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:31.874 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 347, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 573, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 112012, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 482110, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 10, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.058703152, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.023722575, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.027753968, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.004724411, 2024-09-16T00:10:31.875 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 572, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.046263581, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000080880 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 855, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000247123, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000289 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 125936 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.876 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 42, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 75, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 10491, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 47170, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 12, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.007849514, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.001773713, 2024-09-16T00:10:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.005174580, 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000505491, 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 71, 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.017025973, 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000239802 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 58, 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000011065, 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000190 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 4064, 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 11888 2024-09-16T00:10:31.879 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.880 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:10:31.880 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-16T00:10:31.880 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:10:31.880 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.880 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 57, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002334982, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000040964 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:31.881 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000013991, 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000006995 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 8, 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 6, 2024-09-16T00:10:31.882 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 6, 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002009249, 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000334874 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-16T00:10:31.883 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 2, 2024-09-16T00:10:31.884 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:31.884 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:31.884 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-09-16T00:10:31.884 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-16T00:10:31.884 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 11, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 11, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 10, 2024-09-16T00:10:31.885 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 1, 2024-09-16T00:10:31.886 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 1, 2024-09-16T00:10:31.886 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 1, 2024-09-16T00:10:31.886 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 2, 2024-09-16T00:10:31.886 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 15, 2024-09-16T00:10:31.886 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 10, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 2, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-16T00:10:31.887 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 1000, 2024-09-16T00:10:31.888 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 1, 2024-09-16T00:10:31.888 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-16T00:10:31.888 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-16T00:10:31.888 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-16T00:10:31.888 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-09-16T00:10:31.889 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:31.890 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:31.891 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:31.892 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:31.892 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:31.892 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-09-16T00:10:31.892 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:31.892 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-16T00:10:31.893 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 3, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-16T00:10:31.894 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-16T00:10:31.895 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:31.896 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 10, 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-16T00:10:31.897 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 10, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 2, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 2, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-09-16T00:10:31.898 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 4197382, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.019321561, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002146840 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 14, 2024-09-16T00:10:31.899 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 14, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 0, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 13, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 14, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 1, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 11, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 12, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 1, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 1, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 3, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 2, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 49576, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223520 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.900 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 10, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 8, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 29, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.901 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4, 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000808917, 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000202229 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.902 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.001200332, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000600166 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.903 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.904 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.905 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.906 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.907 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.908 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-16T00:10:31.909 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-16T00:10:31.910 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.912 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.913 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 5, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 3, 2024-09-16T00:10:31.914 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 4, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 1, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 596, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:31.915 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:31.916 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:31.917 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:31.918 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 256585, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 102, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 88, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 1, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:31.919 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 176, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 56904, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 225, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:31.920 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 49, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 13705, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 49, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 49, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.234995221, 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004795820 2024-09-16T00:10:31.921 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 49, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 73 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 49, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 49, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 0, 2024-09-16T00:10:31.922 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 9, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 9, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 6, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 0, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-09-16T00:10:31.923 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-16T00:10:31.924 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-16T00:10:31.925 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 91, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 11, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 9, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 9, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 17, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 2 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-16T00:10:31.926 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:31.927 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 781, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1713064, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:31.928 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 781, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 781, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1713064, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.929 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 49, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 13705, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 49, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 13705, 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:31.930 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:31.931 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 49, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 49, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 49, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 49, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:31.932 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.933 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55e814cfe960": { 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 10, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 3078, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 10, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:31.934 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 9, 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 3078, 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:31.935 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:10:32.760 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-16T00:10:32.760 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-16T00:10:32.760 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-16T00:10:32.761 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:32.761 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:32.761 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T00:10:32.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T00:10:33.005 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:10:33.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:33 smithi102 ceph-mon[34458]: pgmap v325: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:33.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:33 smithi031 ceph-mon[31203]: pgmap v325: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:33.537 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:10:33.537 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:33.537 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 359, 2024-09-16T00:10:33.537 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 543, 2024-09-16T00:10:33.538 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 111185, 2024-09-16T00:10:33.538 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 308574, 2024-09-16T00:10:33.538 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 10, 2024-09-16T00:10:33.538 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-16T00:10:33.538 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.058482004, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.021711007, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.028039935, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005530858, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 542, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.048902069, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000090225 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:33.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 829, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000233609, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000281 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 76, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 258, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 26175, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 520795, 2024-09-16T00:10:33.540 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 16, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.019505842, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.009148567, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.008535546, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000203809, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 255, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.031486686, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000123477 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 139, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000034241, 2024-09-16T00:10:33.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000246 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 437376 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 334, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 328, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1669142, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 168789, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.049475939, 2024-09-16T00:10:33.542 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.017813665, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.044765249, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000235471, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 327, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.031438453, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000096142 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 127, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000055154, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000434 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.543 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1646112, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 138704 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 41, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002089170, 2024-09-16T00:10:33.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000050955 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000019101, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000009550 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 2, 2024-09-16T00:10:33.545 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 2, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.001880288, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000940144 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:33.546 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 12, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 12, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 12, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-09-16T00:10:33.547 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 2, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 2, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 2, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 4, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 3, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 1, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 1000, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 1, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-16T00:10:33.548 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:33.549 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:33.550 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 2, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:33.551 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-16T00:10:33.552 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 4, 2024-09-16T00:10:33.553 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 4, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 2, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 2, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-16T00:10:33.554 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 4195780, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.004199705, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001399901 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 15, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 15, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 0, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 13, 2024-09-16T00:10:33.555 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 14, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 1, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 12, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 12, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 0, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 2, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 2, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 50456, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223520 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 6, 2024-09-16T00:10:33.556 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 2, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 17, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 1, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.001880288, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000940144 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.557 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.560 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-16T00:10:33.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.563 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.564 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.565 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.566 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-16T00:10:33.567 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.568 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 0, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 1, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 1, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 599, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:33.569 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:33.570 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:33.571 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 278897, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 97, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:33.572 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 176, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:33.573 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 60032, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 233, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 40, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 11933, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 29, 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 40, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 40, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.227995369, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005699884 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 40, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 63 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 40, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 0, 2024-09-16T00:10:33.575 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 40, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 3, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 7, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 6, 2024-09-16T00:10:33.576 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-16T00:10:33.577 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-16T00:10:33.578 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-16T00:10:33.579 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 91, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 11, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 9, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 9, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 16, 2024-09-16T00:10:33.580 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 1 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-16T00:10:33.581 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:33.582 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 765, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1720794, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 765, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:33.583 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 765, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1720794, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:33.584 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 40, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 11933, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 40, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 11933, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.585 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 40, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 40, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 40, 2024-09-16T00:10:33.586 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 40, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:33.587 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.588 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55cf3aac0960": { 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 4, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1476, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 4, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:33.589 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 3, 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1476, 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:33.590 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:10:33.591 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:33 smithi081 ceph-mon[35060]: pgmap v325: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:34.326 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T00:10:34.326 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T00:10:34.326 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-09-16T00:10:34.327 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:34.327 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:34.327 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T00:10:34.327 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T00:10:34.570 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2096, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2098, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 595201, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 666535, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.176207620, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.057964672, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.044491495, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.059185335, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2096, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.079282888, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000037825 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 164, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 876, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 440746, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.010042854, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006078333, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001915589, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000064277, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 161, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.014084704, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000087482 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 440048 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 245, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 230, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1625548, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 133358, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.045463645, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.015448751, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.045369976, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000021116, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 230, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.018166750, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000078985 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1625280, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 133040 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3562, 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.094673166, 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000026578 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 1, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 4, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 2, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 4195780, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 4195387, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 4 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 3, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 3, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 1, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 1, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 48860, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 215328 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 0, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1651, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 181360, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 48, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 7944, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 176, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 8512, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 13, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 2099, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 21, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 3, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 2099, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2099, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.843982727, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000402088 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2099, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2100 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 2099, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2099, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 1360, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 737, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-16T00:10:35.186 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-16T00:10:35.187 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 91, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 11, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 4, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 4, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:10:35.188 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.189 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 2341, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1995673, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 2341, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2341, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1995673, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.190 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2099, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 1481, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2099, 2024-09-16T00:10:35.191 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1481, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:35.192 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2099, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 2099, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2099, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2099, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:35.193 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.194 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55a984918b40": { 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-16T00:10:35.195 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-16T00:10:35.196 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T00:10:35.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:35 smithi102 ceph-mon[34458]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:35 smithi031 ceph-mon[31203]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:35.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:35 smithi081 ceph-mon[35060]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:36.029 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-16T00:10:36.029 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-16T00:10:36.029 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-16T00:10:36.029 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-16T00:10:36.029 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-16T00:10:36.029 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-16T00:10:36.039 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-16T00:10:36.039 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:10:36.039 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:10:36.061 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:10:36.061 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:10:36.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:10:36.435 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 -- 172.21.15.31:0/3821972080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 msgr2=0x7fc73410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 --2- 172.21.15.31:0/3821972080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73410e600 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ca0 tx=0x7fc72c02a950 comp rx=0 tx=0).stop 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 -- 172.21.15.31:0/3821972080 shutdown_connections 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 --2- 172.21.15.31:0/3821972080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73410e600 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 --2- 172.21.15.31:0/3821972080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7341036c0 0x7fc73410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 --2- 172.21.15.31:0/3821972080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc734102da0 0x7fc734103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 -- 172.21.15.31:0/3821972080 >> 172.21.15.31:0/3821972080 conn(0x7fc7340fc7a0 msgr2=0x7fc7340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 -- 172.21.15.31:0/3821972080 shutdown_connections 2024-09-16T00:10:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.905+0000 7fc73adce640 1 -- 172.21.15.31:0/3821972080 wait complete. 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 Processor -- start 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 -- start start 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734102da0 0x7fc734198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7341036c0 0x7fc7341988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc734111ec0 con 0x7fc73410c180 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc734111d40 con 0x7fc7341036c0 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73adce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112040 con 0x7fc734102da0 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73936b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73936b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45130/0 (socket says 172.21.15.31:45130) 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.906+0000 7fc73936b640 1 -- 172.21.15.31:0/1373496302 learned_addr learned my addr 172.21.15.31:0/1373496302 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc733fff640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7341036c0 0x7fc7341988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc738b6a640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734102da0 0x7fc734198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 -- 172.21.15.31:0/1373496302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734102da0 msgr2=0x7fc734198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734102da0 0x7fc734198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 -- 172.21.15.31:0/1373496302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7341036c0 msgr2=0x7fc7341988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7341036c0 0x7fc7341988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 -- 172.21.15.31:0/1373496302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419d330 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73936b640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009dc0 tx=0x7fc72c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c034040 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc72c030850 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73419d5c0 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0047f0 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.907+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a4f00 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.908+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734103180 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.910+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc72c037020 con 0x7fc73410c180 2024-09-16T00:10:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.910+0000 7fc731ffb640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 0x7fc7240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.910+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7fc72c0f0b40 con 0x7fc73410c180 2024-09-16T00:10:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.910+0000 7fc738b6a640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 0x7fc7240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.911+0000 7fc738b6a640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 0x7fc7240a58f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc728009890 tx=0x7fc728008040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:36.913+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc72c0bd670 con 0x7fc73410c180 2024-09-16T00:10:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.101+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7341996e0 con 0x7fc73410c180 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.102+0000 7fc731ffb640 1 -- 172.21.15.31:0/1373496302 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v91) ==== 81+0+939 (secure 0 0 0) 0x7fc7341996e0 con 0x7fc73410c180 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/540718575 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/797003742 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/797003742 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1530295855 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1530295855 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/741219641 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2677380564 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3308606376 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/639707976 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3011001440 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/639707976 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2347317772 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/108921567 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1147118595 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2022751094 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3296404867 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:37.105 INFO:teuthology.orchestra.run.smithi031.stderr:listed 16 entries 2024-09-16T00:10:37.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 msgr2=0x7fc7240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:37.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 0x7fc7240a58f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc728009890 tx=0x7fc728008040 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 msgr2=0x7fc73419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009dc0 tx=0x7fc72c003f10 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 shutdown_connections 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fc7240a3430 0x7fc7240a58f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.104+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc73410c180 0x7fc73419cc50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.105+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7341036c0 0x7fc7341988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.105+0000 7fc73adce640 1 --2- 172.21.15.31:0/1373496302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734102da0 0x7fc734198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.105+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 >> 172.21.15.31:0/1373496302 conn(0x7fc7340fc7a0 msgr2=0x7fc7340fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.105+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 shutdown_connections 2024-09-16T00:10:37.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:37.105+0000 7fc73adce640 1 -- 172.21.15.31:0/1373496302 wait complete. 2024-09-16T00:10:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:37 smithi102 ceph-mon[34458]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:37.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:37 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1373496302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:37 smithi031 ceph-mon[31203]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:37.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:37 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1373496302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:37.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:37 smithi081 ceph-mon[35060]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:37.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:37 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1373496302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:37.767 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T00:10:37.768 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:10:37.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:10:38.087 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:38.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- 172.21.15.31:0/864908238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 msgr2=0x7fe70810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/864908238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70810eaf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe700009cd0 tx=0x7fe70002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- 172.21.15.31:0/864908238 shutdown_connections 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/864908238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70810eaf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/864908238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe708100ca0 0x7fe708106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/864908238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7081002f0 0x7fe7081006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- 172.21.15.31:0/864908238 >> 172.21.15.31:0/864908238 conn(0x7fe708077210 msgr2=0x7fe7080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- 172.21.15.31:0/864908238 shutdown_connections 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- 172.21.15.31:0/864908238 wait complete. 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 Processor -- start 2024-09-16T00:10:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.558+0000 7fe70f1d3640 1 -- start start 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7081002f0 0x7fe708198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe708100ca0 0x7fe708198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe708111e50 con 0x7fe708106f50 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe708111cd0 con 0x7fe708100ca0 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70f1d3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe708111fd0 con 0x7fe7081002f0 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45156/0 (socket says 172.21.15.31:45156) 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 -- 172.21.15.31:0/2236646580 learned_addr learned my addr 172.21.15.31:0/2236646580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe707fff640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe708100ca0 0x7fe708198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70cf6f640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7081002f0 0x7fe708198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 -- 172.21.15.31:0/2236646580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7081002f0 msgr2=0x7fe708198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7081002f0 0x7fe708198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 -- 172.21.15.31:0/2236646580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe708100ca0 msgr2=0x7fe708198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe708100ca0 0x7fe708198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 -- 172.21.15.31:0/2236646580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70819d3b0 con 0x7fe708106f50 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.559+0000 7fe70d770640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe700033040 tx=0x7fe700003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.560+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700004040 con 0x7fe708106f50 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.560+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe700004690 con 0x7fe708106f50 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.560+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70819d640 con 0x7fe708106f50 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.560+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700030e10 con 0x7fe708106f50 2024-09-16T00:10:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.560+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7081a4f80 con 0x7fe708106f50 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.561+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7081006d0 con 0x7fe708106f50 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.562+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe7000305a0 con 0x7fe708106f50 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.563+0000 7fe705ffb640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 0x7fe6f00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.563+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7fe70002e030 con 0x7fe708106f50 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.563+0000 7fe70cf6f640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 0x7fe6f00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.564+0000 7fe70cf6f640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 0x7fe6f00a58f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800bbe0 tx=0x7fe6f800b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.566+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe70004b060 con 0x7fe708106f50 2024-09-16T00:10:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.753+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe708199760 con 0x7fe708106f50 2024-09-16T00:10:38.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.754+0000 7fe705ffb640 1 -- 172.21.15.31:0/2236646580 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v91) ==== 81+0+939 (secure 0 0 0) 0x7fe7000bde70 con 0x7fe708106f50 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/540718575 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/797003742 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/797003742 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1530295855 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1530295855 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/741219641 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2677380564 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3308606376 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/639707976 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3011001440 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/639707976 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2347317772 2024-09-16T23:50:46.251673+0000 2024-09-16T00:10:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/108921567 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1147118595 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2022751094 2024-09-16T23:51:11.105458+0000 2024-09-16T00:10:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3296404867 2024-09-17T00:00:07.493533+0000 2024-09-16T00:10:38.757 INFO:teuthology.orchestra.run.smithi031.stderr:listed 16 entries 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 msgr2=0x7fe6f00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 0x7fe6f00a58f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800bbe0 tx=0x7fe6f800b390 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 msgr2=0x7fe70819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe700033040 tx=0x7fe700003970 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 shutdown_connections 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe6f00a3430 0x7fe6f00a58f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe708106f50 0x7fe70819ccd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe708100ca0 0x7fe708198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 --2- 172.21.15.31:0/2236646580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7081002f0 0x7fe708198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 >> 172.21.15.31:0/2236646580 conn(0x7fe708077210 msgr2=0x7fe7080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 shutdown_connections 2024-09-16T00:10:38.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:38.756+0000 7fe70f1d3640 1 -- 172.21.15.31:0/2236646580 wait complete. 2024-09-16T00:10:39.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:39 smithi102 ceph-mon[34458]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:39.371 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:39 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2236646580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:39.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:39 smithi031 ceph-mon[31203]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:39.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:39 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2236646580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:39.465 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-09-16T00:10:39.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:10:39.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-16T00:10:39.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[83410]: fuse finished with error 0 and tester_r 0 2024-09-16T00:10:39.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:10:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:39 smithi081 ceph-mon[35060]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:39.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:39 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2236646580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:10:41.370 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:41 smithi102 ceph-mon[34458]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:41 smithi031 ceph-mon[31203]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:41.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:41 smithi081 ceph-mon[35060]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-16T00:10:43.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:43 smithi031 ceph-mon[31203]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:43.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:43 smithi081 ceph-mon[35060]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:43.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:43 smithi102 ceph-mon[34458]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:10:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:45 smithi031 ceph-mon[31203]: pgmap v331: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:45.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:45 smithi081 ceph-mon[35060]: pgmap v331: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:45.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:10:45.596 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T00:10:45.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:10:45.620 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T00:10:45.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:45 smithi102 ceph-mon[34458]: pgmap v331: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:45.676 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T00:10:45.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:10:45.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:10:45.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:10:45.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T00:10:45.849 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:10:45.859 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:10:45.860 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:10:45.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:46.127 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 -- 172.21.15.31:0/2806872454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4106f10 msgr2=0x7f46a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 --2- 172.21.15.31:0/2806872454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4106f10 0x7f46a410ea90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f469c009900 tx=0x7f469c02a410 comp rx=0 tx=0).stop 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 -- 172.21.15.31:0/2806872454 shutdown_connections 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 --2- 172.21.15.31:0/2806872454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4106f10 0x7f46a410ea90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 --2- 172.21.15.31:0/2806872454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4102790 0x7f46a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 --2- 172.21.15.31:0/2806872454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 -- 172.21.15.31:0/2806872454 >> 172.21.15.31:0/2806872454 conn(0x7f46a40fd8c0 msgr2=0x7f46a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 -- 172.21.15.31:0/2806872454 shutdown_connections 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.595+0000 7f46a973a640 1 -- 172.21.15.31:0/2806872454 wait complete. 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 Processor -- start 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 -- start start 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4106f10 0x7f46a419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111e20 con 0x7f46a4102790 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111ca0 con 0x7f46a4101de0 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.596+0000 7f46a973a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111fa0 con 0x7f46a4106f10 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52412/0 (socket says 172.21.15.31:52412) 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 -- 172.21.15.31:0/1403112213 learned_addr learned my addr 172.21.15.31:0/1403112213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a37fe640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4106f10 0x7f46a419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a2ffd640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 -- 172.21.15.31:0/1403112213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4106f10 msgr2=0x7f46a419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4106f10 0x7f46a419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 -- 172.21.15.31:0/1403112213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 msgr2=0x7f46a4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 -- 172.21.15.31:0/1403112213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a419d2f0 con 0x7f46a4102790 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a2ffd640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a27fc640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f469800db80 tx=0x7f469800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698017070 con 0x7f46a4102790 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4698003b70 con 0x7f46a4102790 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a419d5e0 con 0x7f46a4102790 2024-09-16T00:10:46.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980104c0 con 0x7f46a4102790 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.597+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a4ec0 con 0x7f46a4102790 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.599+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a41021c0 con 0x7f46a4102790 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.601+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4698004020 con 0x7f46a4102790 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.601+0000 7f468bfff640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 0x7f468c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.602+0000 7f46a2ffd640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 0x7f468c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.602+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7f46980cfcb0 con 0x7f46a4102790 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.602+0000 7f46a2ffd640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 0x7f468c0a58f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f4694005850 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.606+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f469809c6d0 con 0x7f46a4102790 2024-09-16T00:10:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.820+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a402d7c0 con 0x7f46a4102790 2024-09-16T00:10:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.821+0000 7f468bfff640 1 -- 172.21.15.31:0/1403112213 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11961 (secure 0 0 0) 0x7f469809bfd0 con 0x7f46a4102790 2024-09-16T00:10:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:10:46.823 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"btime":"2024-09-16T00:00:08:496800+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-16T00:00:08.100438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24451,24457,24461,24478,24491]},"id":1}]} 2024-09-16T00:10:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2024-09-16T00:10:46.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.823+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 msgr2=0x7f468c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.823+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 0x7f468c0a58f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46940058c0 tx=0x7f4694005850 comp rx=0 tx=0).stop 2024-09-16T00:10:46.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.823+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 msgr2=0x7f46a4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.823+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f469800db80 tx=0x7f469800b630 comp rx=0 tx=0).stop 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 shutdown_connections 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f468c0a3430 0x7f468c0a58f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a4106f10 0x7f46a419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46a4102790 0x7f46a4198880 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 --2- 172.21.15.31:0/1403112213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a4101de0 0x7f46a4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 >> 172.21.15.31:0/1403112213 conn(0x7f46a40fd8c0 msgr2=0x7f46a40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 shutdown_connections 2024-09-16T00:10:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:46.824+0000 7f46a973a640 1 -- 172.21.15.31:0/1403112213 wait complete. 2024-09-16T00:10:47.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:47 smithi031 ceph-mon[31203]: pgmap v332: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:47.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:47 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/1403112213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:47.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:47.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:47 smithi081 ceph-mon[35060]: pgmap v332: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:47.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:47 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/1403112213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:47.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:47 smithi102 ceph-mon[34458]: pgmap v332: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:47.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:47 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/1403112213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 -- 172.21.15.31:0/1379901729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4101de0 msgr2=0x7fd0b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 --2- 172.21.15.31:0/1379901729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4101de0 0x7fd0b41021c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009900 tx=0x7fd0a402a410 comp rx=0 tx=0).stop 2024-09-16T00:10:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 -- 172.21.15.31:0/1379901729 shutdown_connections 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 --2- 172.21.15.31:0/1379901729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 0x7fd0b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 --2- 172.21.15.31:0/1379901729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102790 0x7fd0b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 --2- 172.21.15.31:0/1379901729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4101de0 0x7fd0b41021c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 -- 172.21.15.31:0/1379901729 >> 172.21.15.31:0/1379901729 conn(0x7fd0b40fd8c0 msgr2=0x7fd0b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 -- 172.21.15.31:0/1379901729 shutdown_connections 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.253+0000 7fd0bb048640 1 -- 172.21.15.31:0/1379901729 wait complete. 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.254+0000 7fd0bb048640 1 Processor -- start 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.254+0000 7fd0bb048640 1 -- start start 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101de0 0x7fd0b41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 0x7fd0b419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4111de0 con 0x7fd0b4102790 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4111c60 con 0x7fd0b4101de0 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0bb048640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4111f60 con 0x7fd0b4106f10 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52428/0 (socket says 172.21.15.31:52428) 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 -- 172.21.15.31:0/3780648861 learned_addr learned my addr 172.21.15.31:0/3780648861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b95e5640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 0x7fd0b419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b8de4640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101de0 0x7fd0b41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 -- 172.21.15.31:0/3780648861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 msgr2=0x7fd0b419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 0x7fd0b419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 -- 172.21.15.31:0/3780648861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101de0 msgr2=0x7fd0b41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101de0 0x7fd0b41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 -- 172.21.15.31:0/3780648861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b419d2a0 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b3fff640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c920 tx=0x7fd09c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.255+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c007c70 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.256+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd09c007e10 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.256+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b419d590 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.256+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c018820 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.256+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a4e70 con 0x7fd0b4102790 2024-09-16T00:10:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.257+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b41021c0 con 0x7fd0b4102790 2024-09-16T00:10:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.258+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd09c0189c0 con 0x7fd0b4102790 2024-09-16T00:10:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.258+0000 7fd0b1ffb640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 0x7fd0a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.259+0000 7fd0b8de4640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 0x7fd0a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.259+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7fd09c0cfd60 con 0x7fd0b4102790 2024-09-16T00:10:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"btime":"2024-09-16T00:00:08:496800+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3861052716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3861052716},{"type":"v1","addr":"172.21.15.31:6835","nonce":3861052716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/735300624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":735300624},{"type":"v1","addr":"172.21.15.31:6837","nonce":735300624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T23:58:11.769531+0000","modified":"2024-09-16T00:00:08.100438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24478,"mds_2":24461,"mds_3":24457,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/193010383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":193010383},{"type":"v1","addr":"172.21.15.31:6839","nonce":193010383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14607":{"gid":14607,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/1341414618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":1341414618},{"type":"v1","addr":"172.21.15.31:6841","nonce":1341414618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/958737504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":958737504},{"type":"v1","addr":"172.21.15.81:6833","nonce":958737504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/4135774449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":4135774449},{"type":"v1","addr":"172.21.15.102:6833","nonce":4135774449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/799148008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":799148008},{"type":"v1","addr":"172.21.15.81:6835","nonce":799148008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.102:6835/2759418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2759418980},{"type":"v1","addr":"172.21.15.102:6835","nonce":2759418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.102:6837/4082772592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":4082772592},{"type":"v1","addr":"172.21.15.102:6837","nonce":4082772592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.81:6837/3184710981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3184710981},{"type":"v1","addr":"172.21.15.81:6837","nonce":3184710981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.102:6839/3686122257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3686122257},{"type":"v1","addr":"172.21.15.102:6839","nonce":3686122257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6839/471643993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":471643993},{"type":"v1","addr":"172.21.15.81:6839","nonce":471643993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24451,24457,24461,24478,24491]},"id":1}]} 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.259+0000 7fd0b8de4640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 0x7fd0a00a5870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4031440 tx=0x7fd0a4005b50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.262+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd09c014030 con 0x7fd0b4102790 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.471+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b402d7c0 con 0x7fd0b4102790 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.472+0000 7fd0b1ffb640 1 -- 172.21.15.31:0/3780648861 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11961 (secure 0 0 0) 0x7fd09c09c780 con 0x7fd0b4102790 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2024-09-16T00:10:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 msgr2=0x7fd0a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 0x7fd0a00a5870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4031440 tx=0x7fd0a4005b50 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 msgr2=0x7fd0b4198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c920 tx=0x7fd09c00cde0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 shutdown_connections 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd0a00a33b0 0x7fd0a00a5870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b4106f10 0x7fd0b419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b4102790 0x7fd0b4198830 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 --2- 172.21.15.31:0/3780648861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101de0 0x7fd0b41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 >> 172.21.15.31:0/3780648861 conn(0x7fd0b40fd8c0 msgr2=0x7fd0b40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 shutdown_connections 2024-09-16T00:10:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:48.474+0000 7fd0bb048640 1 -- 172.21.15.31:0/3780648861 wait complete. 2024-09-16T00:10:49.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:10:49.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:49 smithi031 ceph-mon[31203]: pgmap v333: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:49.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:49 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3780648861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:49.410 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:49.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:49 smithi081 ceph-mon[35060]: pgmap v333: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:49.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:49 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3780648861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:49.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:49 smithi102 ceph-mon[34458]: pgmap v333: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:49.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:49 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3780648861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 -- 172.21.15.31:0/1239601531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964102d40 msgr2=0x7f4964103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 --2- 172.21.15.31:0/1239601531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964102d40 0x7f4964103120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4954009900 tx=0x7f495402a480 comp rx=0 tx=0).stop 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 -- 172.21.15.31:0/1239601531 shutdown_connections 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 --2- 172.21.15.31:0/1239601531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 --2- 172.21.15.31:0/1239601531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964103660 0x7f496410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 --2- 172.21.15.31:0/1239601531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964102d40 0x7f4964103120 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 -- 172.21.15.31:0/1239601531 >> 172.21.15.31:0/1239601531 conn(0x7f49640fc7a0 msgr2=0x7f49640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 -- 172.21.15.31:0/1239601531 shutdown_connections 2024-09-16T00:10:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.870+0000 7f4969bfb640 1 -- 172.21.15.31:0/1239601531 wait complete. 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 Processor -- start 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 -- start start 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 0x7f4964198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111e60 con 0x7f4964103660 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111ce0 con 0x7f496410c170 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4969bfb640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111fe0 con 0x7f4964102d40 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4962ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4962ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52440/0 (socket says 172.21.15.31:52440) 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.871+0000 7f4962ffd640 1 -- 172.21.15.31:0/3679715008 learned_addr learned my addr 172.21.15.31:0/3679715008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4963fff640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f49637fe640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 0x7f4964198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 -- 172.21.15.31:0/3679715008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 msgr2=0x7f4964198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 0x7f4964198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 -- 172.21.15.31:0/3679715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 msgr2=0x7f496419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 -- 172.21.15.31:0/3679715008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496419d3d0 con 0x7f4964103660 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4963fff640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f49637fe640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 0x7f4964198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4962ffd640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f495800b3f0 tx=0x7f495800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958015020 con 0x7f4964103660 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496419d6c0 con 0x7f4964103660 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4958011ce0 con 0x7f4964103660 2024-09-16T00:10:49.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958010d80 con 0x7f4964103660 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.872+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a4fa0 con 0x7f4964103660 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.873+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4964103120 con 0x7f4964103660 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.874+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4958018020 con 0x7f4964103660 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.875+0000 7f4960ff9640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 0x7f494c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.875+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7f49580cf510 con 0x7f4964103660 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.875+0000 7f49637fe640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 0x7f494c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.876+0000 7f49637fe640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 0x7f494c0a5870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4954000c00 tx=0x7f4954031400 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:49.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:49.880+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f495801f020 con 0x7f4964103660 2024-09-16T00:10:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.058+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4964107240 con 0x7f4964103660 2024-09-16T00:10:50.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.060+0000 7f4960ff9640 1 -- 172.21.15.31:0/3679715008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23039 (secure 0 0 0) 0x7f495809bfe0 con 0x7f4964103660 2024-09-16T00:10:50.062 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T00:10:50.062 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"21a99a0a-73bd-11ef-bceb-c7b262605968","created":"2024-09-15T23:50:23.938656+0000","modified":"2024-09-16T00:10:14.097922+0000","last_up_change":"2024-09-15T23:57:18.958661+0000","last_in_change":"2024-09-15T23:56:54.758649+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T23:53:27.616831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T23:58:07.909795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T23:58:10.116305+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b544a11-7388-485f-9e4b-49e5bfa8459b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6803","nonce":1628537421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6805","nonce":1628537421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6809","nonce":1628537421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1628537421},{"type":"v1","addr":"172.21.15.31:6807","nonce":1628537421}]},"public_addr":"172.21.15.31:6803/1628537421","cluster_addr":"172.21.15.31:6805/1628537421","heartbeat_back_addr":"172.21.15.31:6809/1628537421","heartbeat_front_addr":"172.21.15.31:6807/1628537421","state":["exists","up"]},{"osd":1,"uuid":"cc0ccb76-ecd7-4c5b-b4f4-f282127b892a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6801","nonce":796455995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6803","nonce":796455995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6807","nonce":796455995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":796455995},{"type":"v1","addr":"172.21.15.81:6805","nonce":796455995}]},"public_addr":"172.21.15.81:6801/796455995","cluster_addr":"172.21.15.81:6803/796455995","heartbeat_back_addr":"172.21.15.81:6807/796455995","heartbeat_front_addr":"172.21.15.81:6805/796455995","state":["exists","up"]},{"osd":2,"uuid":"16d860f1-d8da-4d5d-ae50-10e77a4c4b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6801","nonce":3162988112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6803","nonce":3162988112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6807","nonce":3162988112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":3162988112},{"type":"v1","addr":"172.21.15.102:6805","nonce":3162988112}]},"public_addr":"172.21.15.102:6801/3162988112","cluster_addr":"172.21.15.102:6803/3162988112","heartbeat_back_addr":"172.21.15.102:6807/3162988112","heartbeat_front_addr":"172.21.15.102:6805/3162988112","state":["exists","up"]},{"osd":3,"uuid":"7e149e85-40e7-4d2b-9ce5-fa2e3cc6c747","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6811","nonce":487246469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6813","nonce":487246469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6817","nonce":487246469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":487246469},{"type":"v1","addr":"172.21.15.31:6815","nonce":487246469}]},"public_addr":"172.21.15.31:6811/487246469","cluster_addr":"172.21.15.31:6813/487246469","heartbeat_back_addr":"172.21.15.31:6817/487246469","heartbeat_front_addr":"172.21.15.31:6815/487246469","state":["exists","up"]},{"osd":4,"uuid":"f1085071-44fa-4277-b09b-7e5ef3b45330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6809","nonce":3997776531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6811","nonce":3997776531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6815","nonce":3997776531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3997776531},{"type":"v1","addr":"172.21.15.81:6813","nonce":3997776531}]},"public_addr":"172.21.15.81:6809/3997776531","cluster_addr":"172.21.15.81:6811/3997776531","heartbeat_back_addr":"172.21.15.81:6815/3997776531","heartbeat_front_addr":"172.21.15.81:6813/3997776531","state":["exists","up"]},{"osd":5,"uuid":"010742fa-38fa-4483-9f2f-173c0fd11a06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6809","nonce":3259467463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6811","nonce":3259467463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6815","nonce":3259467463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":3259467463},{"type":"v1","addr":"172.21.15.102:6813","nonce":3259467463}]},"public_addr":"172.21.15.102:6809/3259467463","cluster_addr":"172.21.15.102:6811/3259467463","heartbeat_back_addr":"172.21.15.102:6815/3259467463","heartbeat_front_addr":"172.21.15.102:6813/3259467463","state":["exists","up"]},{"osd":6,"uuid":"d7c78c23-60ea-4bb6-b269-9d236295e6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6819","nonce":515354645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6821","nonce":515354645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6825","nonce":515354645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":515354645},{"type":"v1","addr":"172.21.15.31:6823","nonce":515354645}]},"public_addr":"172.21.15.31:6819/515354645","cluster_addr":"172.21.15.31:6821/515354645","heartbeat_back_addr":"172.21.15.31:6825/515354645","heartbeat_front_addr":"172.21.15.31:6823/515354645","state":["exists","up"]},{"osd":7,"uuid":"f270c537-d25a-4ee7-87a0-b18f67a00a2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6817","nonce":326400233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6819","nonce":326400233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6823","nonce":326400233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":326400233},{"type":"v1","addr":"172.21.15.81:6821","nonce":326400233}]},"public_addr":"172.21.15.81:6817/326400233","cluster_addr":"172.21.15.81:6819/326400233","heartbeat_back_addr":"172.21.15.81:6823/326400233","heartbeat_front_addr":"172.21.15.81:6821/326400233","state":["exists","up"]},{"osd":8,"uuid":"530bcf2d-1d5c-4ad2-bba2-fad10d3e581e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6817","nonce":3203076122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6819","nonce":3203076122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6823","nonce":3203076122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3203076122},{"type":"v1","addr":"172.21.15.102:6821","nonce":3203076122}]},"public_addr":"172.21.15.102:6817/3203076122","cluster_addr":"172.21.15.102:6819/3203076122","heartbeat_back_addr":"172.21.15.102:6823/3203076122","heartbeat_front_addr":"172.21.15.102:6821/3203076122","state":["exists","up"]},{"osd":9,"uuid":"d1bf1827-c5c0-4b3b-9f97-f7eb1679d742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6827","nonce":1878664513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6829","nonce":1878664513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6833","nonce":1878664513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1878664513},{"type":"v1","addr":"172.21.15.31:6831","nonce":1878664513}]},"public_addr":"172.21.15.31:6827/1878664513","cluster_addr":"172.21.15.31:6829/1878664513","heartbeat_back_addr":"172.21.15.31:6833/1878664513","heartbeat_front_addr":"172.21.15.31:6831/1878664513","state":["exists","up"]},{"osd":10,"uuid":"a086c66d-46cc-4b3a-a8cf-7022a6acf89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6825","nonce":397827149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6827","nonce":397827149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6831","nonce":397827149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":397827149},{"type":"v1","addr":"172.21.15.81:6829","nonce":397827149}]},"public_addr":"172.21.15.81:6825/397827149","cluster_addr":"172.21.15.81:6827/397827149","heartbeat_back_addr":"172.21.15.81:6831/397827149","heartbeat_front_addr":"172.21.15.81:6829/397827149","state":["exists","up"]},{"osd":11,"uuid":"3eead23d-bbb5-4fd5-adc7-3330798d55f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6825","nonce":4272602493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6827","nonce":4272602493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6831","nonce":4272602493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":4272602493},{"type":"v1","addr":"172.21.15.102:6829","nonce":4272602493}]},"public_addr":"172.21.15.102:6825/4272602493","cluster_addr":"172.21.15.102:6827/4272602493","heartbeat_back_addr":"172.21.15.102:6831/4272602493","heartbeat_front_addr":"172.21.15.102:6829/4272602493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:52:35.911767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:00.986680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:25.434916+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:53:50.952087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:17.240357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:54:43.094157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:09.241804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:34.908588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:55:59.411023+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:24.925592+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:56:50.946645+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T23:57:16.978261+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/540718575":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6801/797003742":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6800/797003742":"2024-09-17T00:00:07.493533+0000","172.21.15.31:6801/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/1530295855":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/741219641":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/2677380564":"2024-09-17T00:00:07.493533+0000","172.21.15.31:0/3308606376":"2024-09-16T23:50:46.251673+0000","172.21.15.31:6800/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3011001440":"2024-09-16T23:51:11.105458+0000","172.21.15.31:6801/639707976":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/2347317772":"2024-09-16T23:50:46.251673+0000","172.21.15.31:0/108921567":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/1147118595":"2024-09-17T00:00:07.493533+0000","172.21.15.31:0/2022751094":"2024-09-16T23:51:11.105458+0000","172.21.15.31:0/3296404867":"2024-09-17T00:00:07.493533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:10:50.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.062+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 msgr2=0x7f494c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.062+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 0x7f494c0a5870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4954000c00 tx=0x7f4954031400 comp rx=0 tx=0).stop 2024-09-16T00:10:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.062+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 msgr2=0x7f4964198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.062+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f495800b3f0 tx=0x7f495800b8b0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 shutdown_connections 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f494c0a33b0 0x7f494c0a5870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496410c170 0x7f496419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4964103660 0x7f4964198960 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 --2- 172.21.15.31:0/3679715008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964102d40 0x7f4964198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 >> 172.21.15.31:0/3679715008 conn(0x7f49640fc7a0 msgr2=0x7f49640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 shutdown_connections 2024-09-16T00:10:50.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:50.063+0000 7f4969bfb640 1 -- 172.21.15.31:0/3679715008 wait complete. 2024-09-16T00:10:50.426 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:50 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3679715008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:50.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:50 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3679715008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:50.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:50 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3679715008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:50.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T00:10:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:51.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-mon[31203]: pgmap v334: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:51.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-mon[35060]: pgmap v334: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 -- 172.21.15.31:0/3819303931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 msgr2=0x7f558810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 --2- 172.21.15.31:0/3819303931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f558810bb50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5578009900 tx=0x7f557802a480 comp rx=0 tx=0).stop 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 -- 172.21.15.31:0/3819303931 shutdown_connections 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 --2- 172.21.15.31:0/3819303931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 0x7f558810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 --2- 172.21.15.31:0/3819303931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f558810bb50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 --2- 172.21.15.31:0/3819303931 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f5588103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 -- 172.21.15.31:0/3819303931 >> 172.21.15.31:0/3819303931 conn(0x7f55880fc7a0 msgr2=0x7f55880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 -- 172.21.15.31:0/3819303931 shutdown_connections 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.557+0000 7f55906c8640 1 -- 172.21.15.31:0/3819303931 wait complete. 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 Processor -- start 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 -- start start 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f55881983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 0x7f558819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111e20 con 0x7f5588103660 2024-09-16T00:10:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111ca0 con 0x7f5588102d40 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f55906c8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111fa0 con 0x7f558810c170 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f558dc63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f558dc63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52462/0 (socket says 172.21.15.31:52462) 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.558+0000 7f558dc63640 1 -- 172.21.15.31:0/546162642 learned_addr learned my addr 172.21.15.31:0/546162642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558ec65640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 0x7f558819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558e464640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f55881983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 -- 172.21.15.31:0/546162642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 msgr2=0x7f558819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 0x7f558819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 -- 172.21.15.31:0/546162642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 msgr2=0x7f55881983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f55881983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 -- 172.21.15.31:0/546162642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558819d350 con 0x7f5588103660 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558e464640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f55881983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f558dc63640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5578005940 tx=0x7f55780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557803f030 con 0x7f5588103660 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558819d5e0 con 0x7f5588103660 2024-09-16T00:10:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5578003cb0 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578035070 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.559+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881a4f20 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.560+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558806b070 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.562+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5578042030 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.562+0000 7f557f7fe640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 0x7f55740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.562+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8463+0+0 (secure 0 0 0) 0x7f55780f0240 con 0x7f5588103660 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.562+0000 7f558e464640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 0x7f55740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.563+0000 7f558e464640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 0x7f55740a5870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5584009b40 tx=0x7f5584005e50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.567+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55780bcc00 con 0x7f5588103660 2024-09-16T00:10:51.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-mon[34458]: pgmap v334: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-16T00:10:51.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.779+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55881071e0 con 0x7f5588103660 2024-09-16T00:10:51.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.782+0000 7f557f7fe640 1 -- 172.21.15.31:0/546162642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f55780bc500 con 0x7f5588103660 2024-09-16T00:10:51.782 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:10:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.785+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 msgr2=0x7f55740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.785+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 0x7f55740a5870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5584009b40 tx=0x7f5584005e50 comp rx=0 tx=0).stop 2024-09-16T00:10:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.785+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 msgr2=0x7f55881988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:51.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.785+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5578005940 tx=0x7f55780042f0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 shutdown_connections 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f55740a33b0 0x7f55740a5870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f558810c170 0x7f558819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5588103660 0x7f55881988e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 --2- 172.21.15.31:0/546162642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5588102d40 0x7f55881983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 >> 172.21.15.31:0/546162642 conn(0x7f55880fc7a0 msgr2=0x7f55880fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 shutdown_connections 2024-09-16T00:10:51.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:51.786+0000 7f55906c8640 1 -- 172.21.15.31:0/546162642 wait complete. 2024-09-16T00:10:52.120 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i[64765]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i[64765]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-mds[64769]: -- 172.21.15.102:0/1647064090 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2845a7860 con 0x55f284585000 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i[64765]: starting mds.i at 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-mds[64166]: -- 172.21.15.102:0/2984845287 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a9c753860 con 0x563a9c730800 2024-09-16T00:10:52.121 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: starting mds.f at 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-mds[63865]: -- 172.21.15.102:0/2978453842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560046d1860 con 0x5560046af000 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: starting mds.l at 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-mds[64466]: -- 172.21.15.102:0/578069514 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9b6231c20 con 0x55f9b620f000 2024-09-16T00:10:52.122 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:10:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: starting mds.c at 2024-09-16T00:10:52.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a[31199]: 2024-09-16T00:10:51.780+0000 7f9d388ac640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-mds[72631]: -- 172.21.15.31:0/1249254333 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc84855860 con 0x55bc84832800 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:10:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: starting mds.a at 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.145 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: starting mds.h at 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-mds[64733]: -- 172.21.15.81:0/1568046220 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d39c73860 con 0x559d39c51000 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: starting mds.b at 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.146 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-mds[63816]: -- 172.21.15.81:0/1029283918 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55997e30f860 con 0x55997e2ed000 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: starting mds.k at 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: ignoring --setuser ceph since I am not root 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: ignoring --setgroup ceph since I am not root 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-mds[64114]: -- 172.21.15.81:0/1676687321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbc9a13860 con 0x55bbc99f0800 2024-09-16T00:10:52.147 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:10:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: starting mds.e at 2024-09-16T00:10:52.487 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:10:52.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:10:52.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:10:52.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:10:52.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:10:52.488 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:52 smithi031 ceph-mon[31203]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:10:52.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:10:52.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:52 smithi081 ceph-mon[35060]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:10:52.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:10:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:10:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:10:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/546162642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:10:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:10:52.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:52 smithi102 ceph-mon[34458]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:10:52.780 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: pgmap v335: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6838/2888266800,v1:172.21.15.31:6839/2888266800] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6832/1418689174,v1:172.21.15.102:6834/1418689174] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6833/1764898051,v1:172.21.15.102:6835/1764898051] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.31:6840/2601744780,v1:172.21.15.31:6841/2601744780] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6832/4106164082,v1:172.21.15.81:6833/4106164082] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6836/427310705,v1:172.21.15.102:6838/427310705] up:boot 2024-09-16T00:10:53.184 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6834/1422848108,v1:172.21.15.81:6835/1422848108] up:boot 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.102:6837/3604128054,v1:172.21.15.102:6839/3604128054] up:boot 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6837/2127225798,v1:172.21.15.81:6839/2127225798] up:boot 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: mds.? [v2:172.21.15.81:6836/3396595242,v1:172.21.15.81:6838/3396595242] up:boot 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:53.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:53.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:53 smithi031 ceph-mon[31203]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:53.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 -- 172.21.15.31:0/571079889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744106a60 msgr2=0x7f2744106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 --2- 172.21.15.31:0/571079889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744106a60 0x7f2744106e40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2740009930 tx=0x7f274002a420 comp rx=0 tx=0).stop 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 -- 172.21.15.31:0/571079889 shutdown_connections 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 --2- 172.21.15.31:0/571079889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744101400 0x7f274410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 --2- 172.21.15.31:0/571079889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744100a60 0x7f2744100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 --2- 172.21.15.31:0/571079889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744106a60 0x7f2744106e40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 -- 172.21.15.31:0/571079889 >> 172.21.15.31:0/571079889 conn(0x7f27440fc780 msgr2=0x7f27440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 -- 172.21.15.31:0/571079889 shutdown_connections 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 -- 172.21.15.31:0/571079889 wait complete. 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.267+0000 7f274baec640 1 Processor -- start 2024-09-16T00:10:53.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 -- start start 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744100a60 0x7f27440fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744106a60 0x7f27440fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2744110d80 con 0x7f2744101400 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2744110c00 con 0x7f2744100a60 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274baec640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2744110f00 con 0x7f2744106a60 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49744/0 (socket says 172.21.15.31:49744) 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 -- 172.21.15.31:0/2084982409 learned_addr learned my addr 172.21.15.31:0/2084982409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749888640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744100a60 0x7f27440fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f274a089640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744106a60 0x7f27440fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 -- 172.21.15.31:0/2084982409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744106a60 msgr2=0x7f27440fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744106a60 0x7f27440fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 -- 172.21.15.31:0/2084982409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744100a60 msgr2=0x7f27440fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744100a60 0x7f27440fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 -- 172.21.15.31:0/2084982409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441a68c0 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.268+0000 7f2749087640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f273400b4a0 tx=0x7f273400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.269+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734014070 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.269+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2734010ac0 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.269+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441a6bb0 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.269+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734017c30 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.269+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a70f0 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.270+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744100ec0 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.271+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2734017070 con 0x7f2744101400 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.271+0000 7f273affd640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 0x7f27300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.271+0000 7f2749888640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 0x7f27300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.271+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8849+0+0 (secure 0 0 0) 0x7f27340cf4e0 con 0x7f2744101400 2024-09-16T00:10:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.272+0000 7f2749888640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 0x7f27300a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2740006010 tx=0x7f2740000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.276+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f273409be80 con 0x7f2744101400 2024-09-16T00:10:53.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:53.489+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2744109fe0 con 0x7f2744101400 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: pgmap v335: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6838/2888266800,v1:172.21.15.31:6839/2888266800] up:boot 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6832/1418689174,v1:172.21.15.102:6834/1418689174] up:boot 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6833/1764898051,v1:172.21.15.102:6835/1764898051] up:boot 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.31:6840/2601744780,v1:172.21.15.31:6841/2601744780] up:boot 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6832/4106164082,v1:172.21.15.81:6833/4106164082] up:boot 2024-09-16T00:10:53.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6836/427310705,v1:172.21.15.102:6838/427310705] up:boot 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6834/1422848108,v1:172.21.15.81:6835/1422848108] up:boot 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.102:6837/3604128054,v1:172.21.15.102:6839/3604128054] up:boot 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6837/2127225798,v1:172.21.15.81:6839/2127225798] up:boot 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: mds.? [v2:172.21.15.81:6836/3396595242,v1:172.21.15.81:6838/3396595242] up:boot 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:53.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:53.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:53.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:53.559 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:53 smithi081 ceph-mon[35060]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: pgmap v335: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6838/2888266800,v1:172.21.15.31:6839/2888266800] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6832/1418689174,v1:172.21.15.102:6834/1418689174] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6833/1764898051,v1:172.21.15.102:6835/1764898051] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.31:6840/2601744780,v1:172.21.15.31:6841/2601744780] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6832/4106164082,v1:172.21.15.81:6833/4106164082] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6836/427310705,v1:172.21.15.102:6838/427310705] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6834/1422848108,v1:172.21.15.81:6835/1422848108] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.102:6837/3604128054,v1:172.21.15.102:6839/3604128054] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6837/2127225798,v1:172.21.15.81:6839/2127225798] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: mds.? [v2:172.21.15.81:6836/3396595242,v1:172.21.15.81:6838/3396595242] up:boot 2024-09-16T00:10:53.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:53 smithi102 ceph-mon[34458]: from='mgr.15216 172.21.15.31:0/147191621' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:54.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.151+0000 7f273affd640 1 -- 172.21.15.31:0/2084982409 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f273409b780 con 0x7f2744101400 2024-09-16T00:10:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 msgr2=0x7f27300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 0x7f27300a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2740006010 tx=0x7f2740000bc0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 msgr2=0x7f27440fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f273400b4a0 tx=0x7f273400b960 comp rx=0 tx=0).stop 2024-09-16T00:10:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 shutdown_connections 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7f27300a33b0 0x7f27300a5870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2744106a60 0x7f27440fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2744101400 0x7f27440fd440 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 --2- 172.21.15.31:0/2084982409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2744100a60 0x7f27440fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 >> 172.21.15.31:0/2084982409 conn(0x7f27440fc780 msgr2=0x7f274406aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 shutdown_connections 2024-09-16T00:10:54.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:54.154+0000 7f274baec640 1 -- 172.21.15.31:0/2084982409 wait complete. 2024-09-16T00:10:54.480 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:54 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:10:54.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:54 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:10:54.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:54 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:10:54.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:10:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:55.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: pgmap v337: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:55.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:10:55.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:10:55.458 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: Cluster is now healthy 2024-09-16T00:10:55.459 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:10:55.459 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:55 smithi081 ceph-mon[35060]: fsmap 12 up:standby 2024-09-16T00:10:55.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: pgmap v337: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:55.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:10:55.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:10:55.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: Cluster is now healthy 2024-09-16T00:10:55.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:10:55.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:55 smithi031 ceph-mon[31203]: fsmap 12 up:standby 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 -- 172.21.15.31:0/373147058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 msgr2=0x7fe92010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 --2- 172.21.15.31:0/373147058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe92010ec40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe918009930 tx=0x7fe91802a440 comp rx=0 tx=0).stop 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 -- 172.21.15.31:0/373147058 shutdown_connections 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 --2- 172.21.15.31:0/373147058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe92010ec40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 --2- 172.21.15.31:0/373147058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9201036e0 0x7fe920107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 --2- 172.21.15.31:0/373147058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe920102d30 0x7fe920103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 -- 172.21.15.31:0/373147058 >> 172.21.15.31:0/373147058 conn(0x7fe9200fc7d0 msgr2=0x7fe9200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 -- 172.21.15.31:0/373147058 shutdown_connections 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.570+0000 7fe92762e640 1 -- 172.21.15.31:0/373147058 wait complete. 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 Processor -- start 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 -- start start 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe920102d30 0x7fe920100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9201036e0 0x7fe920100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe920112010 con 0x7fe9201082a0 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe920111e90 con 0x7fe920102d30 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe92762e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe920112190 con 0x7fe9201036e0 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49758/0 (socket says 172.21.15.31:49758) 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 -- 172.21.15.31:0/3167555799 learned_addr learned my addr 172.21.15.31:0/3167555799 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe9253ca640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe920102d30 0x7fe920100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe924bc9640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9201036e0 0x7fe920100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 -- 172.21.15.31:0/3167555799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9201036e0 msgr2=0x7fe920100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9201036e0 0x7fe920100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 -- 172.21.15.31:0/3167555799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe920102d30 msgr2=0x7fe920100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.571+0000 7fe925bcb640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe920102d30 0x7fe920100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe925bcb640 1 -- 172.21.15.31:0/3167555799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9201a6a90 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe925bcb640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe918004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91803f030 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9201a6d20 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9180035f0 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918035070 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.572+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201a7200 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.573+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe920103190 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.574+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe918003b10 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.575+0000 7fe91e7fc640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 0x7fe9080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.575+0000 7fe9253ca640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 0x7fe9080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.575+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8849+0+0 (secure 0 0 0) 0x7fe9180f0ac0 con 0x7fe9201082a0 2024-09-16T00:10:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.575+0000 7fe9253ca640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 0x7fe9080a5870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe91000a400 tx=0x7fe91000d040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.579+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9180bd2f0 con 0x7fe9201082a0 2024-09-16T00:10:55.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: pgmap v337: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:55.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:10:55.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:10:55.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: Cluster is now healthy 2024-09-16T00:10:55.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/2084982409' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:10:55.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:55 smithi102 ceph-mon[34458]: fsmap 12 up:standby 2024-09-16T00:10:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:55.756+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe92010a2b0 con 0x7fe9201082a0 2024-09-16T00:10:56.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.155+0000 7fe91e7fc640 1 -- 172.21.15.31:0/3167555799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fe9180bcbf0 con 0x7fe9201082a0 2024-09-16T00:10:56.156 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 msgr2=0x7fe9080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 0x7fe9080a5870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe91000a400 tx=0x7fe91000d040 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 msgr2=0x7fe920101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe918004270 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 shutdown_connections 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fe9080a33b0 0x7fe9080a5870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9201082a0 0x7fe920101410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9201036e0 0x7fe920100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 --2- 172.21.15.31:0/3167555799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe920102d30 0x7fe920100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 >> 172.21.15.31:0/3167555799 conn(0x7fe9200fc7d0 msgr2=0x7fe920105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 shutdown_connections 2024-09-16T00:10:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:56.157+0000 7fe92762e640 1 -- 172.21.15.31:0/3167555799 wait complete. 2024-09-16T00:10:56.471 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:56 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:56.489 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:56 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:56.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:56 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:56.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:10:57.103 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:57 smithi031 ceph-mon[31203]: pgmap v338: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:57 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:57.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:57 smithi031 ceph-mon[31203]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:10:57.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:57 smithi081 ceph-mon[35060]: pgmap v338: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:57.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:57 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:57.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:57 smithi081 ceph-mon[35060]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:10:57.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:57 smithi102 ceph-mon[34458]: pgmap v338: 97 pgs: 97 active+clean; 1.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-16T00:10:57.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:57 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/3167555799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:57.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:57 smithi102 ceph-mon[34458]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.631+0000 7fd1a82cd640 1 -- 172.21.15.31:0/2570393121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 msgr2=0x7fd1a010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.631+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/2570393121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a010e610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd198009ca0 tx=0x7fd19802a950 comp rx=0 tx=0).stop 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 -- 172.21.15.31:0/2570393121 shutdown_connections 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/2570393121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a010e610 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/2570393121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0103660 0x7fd1a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/2570393121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0102d40 0x7fd1a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 -- 172.21.15.31:0/2570393121 >> 172.21.15.31:0/2570393121 conn(0x7fd1a00fc7a0 msgr2=0x7fd1a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 -- 172.21.15.31:0/2570393121 shutdown_connections 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 -- 172.21.15.31:0/2570393121 wait complete. 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 Processor -- start 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.632+0000 7fd1a82cd640 1 -- start start 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0102d40 0x7fd1a0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 0x7fd1a0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0111e00 con 0x7fd1a010c170 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0111c80 con 0x7fd1a0103660 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0111f80 con 0x7fd1a0102d40 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a686a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 0x7fd1a0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a686a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49776/0 (socket says 172.21.15.31:49776) 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a686a640 1 -- 172.21.15.31:0/106179078 learned_addr learned my addr 172.21.15.31:0/106179078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a6069640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0102d40 0x7fd1a0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 -- 172.21.15.31:0/106179078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0102d40 msgr2=0x7fd1a0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0102d40 0x7fd1a0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 -- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 msgr2=0x7fd1a019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 -- 172.21.15.31:0/106179078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a019d2f0 con 0x7fd1a0103660 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a686a640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a5868640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 0x7fd1a0198880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d650 tx=0x7fd19000db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017070 con 0x7fd1a0103660 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1900054d0 con 0x7fd1a0103660 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.633+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a019d5e0 con 0x7fd1a0103660 2024-09-16T00:10:57.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.634+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900047a0 con 0x7fd1a0103660 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.634+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01a4ec0 con 0x7fd1a0103660 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.634+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a0103120 con 0x7fd1a0103660 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.636+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd190005020 con 0x7fd1a0103660 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.636+0000 7fd1977fe640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 0x7fd18c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.636+0000 7fd1a6069640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 0x7fd18c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.636+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8353+0+0 (secure 0 0 0) 0x7fd1900cfae0 con 0x7fd1a0103660 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.637+0000 7fd1a6069640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 0x7fd18c0a5870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009bd0 tx=0x7fd19c005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:57.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.640+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd190020020 con 0x7fd1a0103660 2024-09-16T00:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:57.815+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1a01996a0 con 0x7fd1a0103660 2024-09-16T00:10:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.165+0000 7fd1977fe640 1 -- 172.21.15.31:0/106179078 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7fd1a01996a0 con 0x7fd1a0103660 2024-09-16T00:10:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 msgr2=0x7fd18c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 0x7fd18c0a5870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009bd0 tx=0x7fd19c005f90 comp rx=0 tx=0).stop 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 msgr2=0x7fd1a0198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 0x7fd1a0198880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d650 tx=0x7fd19000db10 comp rx=0 tx=0).stop 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 shutdown_connections 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:6800/225135964,v1:172.21.15.31:6801/225135964] conn(0x7fd18c0a33b0 0x7fd18c0a5870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1a010c170 0x7fd1a019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a0103660 0x7fd1a0198880 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 --2- 172.21.15.31:0/106179078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1a0102d40 0x7fd1a0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 >> 172.21.15.31:0/106179078 conn(0x7fd1a00fc7a0 msgr2=0x7fd1a00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 shutdown_connections 2024-09-16T00:10:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T00:10:58.169+0000 7fd1a82cd640 1 -- 172.21.15.31:0/106179078 wait complete. 2024-09-16T00:10:58.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:58 smithi031 ceph-mon[31203]: from='client.? 172.21.15.31:0/106179078' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:58 smithi031 ceph-mon[31203]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:58 smithi081 ceph-mon[35060]: from='client.? 172.21.15.31:0/106179078' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:58 smithi081 ceph-mon[35060]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:58 smithi102 ceph-mon[34458]: from='client.? 172.21.15.31:0/106179078' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:58 smithi102 ceph-mon[34458]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:10:58.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:10:58.828 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:10:58.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:10:58.896 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:10:58.896 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:10:58.913 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:10:58.931 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:10:58.954 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:10:58.955 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T00:10:58.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:10:59.163 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 systemd[1]: Stopping Ceph mon.a for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:10:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 ceph-mon[31203]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 ceph-mon[31203]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 ceph-mon[31203]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:10:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a[31199]: 2024-09-16T00:10:59.269+0000 7f9d3e0b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:10:59.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a[31199]: 2024-09-16T00:10:59.269+0000 7f9d3e0b7640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:10:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:59 smithi081 ceph-mon[35060]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:59 smithi081 ceph-mon[35060]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:59.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:10:59 smithi081 ceph-mon[35060]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:10:59.620 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:59 smithi102 ceph-mon[34458]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:59.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:59 smithi102 ceph-mon[34458]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:10:59.621 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:10:59 smithi102 ceph-mon[34458]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:10:59.813 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 podman[89640]: 2024-09-16 00:10:59.525584566 +0000 UTC m=+0.368695550 container died 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:00.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 podman[89640]: 2024-09-16 00:10:59.896100196 +0000 UTC m=+0.739211179 container cleanup 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T00:11:00.115 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:10:59 smithi031 bash[89640]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:11:00.381 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 16 00:11:00 smithi031 podman[89652]: 2024-09-16 00:11:00.114436083 +0000 UTC m=+0.585603921 container remove 6c2bf311443b385e5959bfeb1108dafae77b36f2101b0ee05398e3512109b501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:11:00.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:11:00.194+0000 7f6aaa855640 -1 client.15234: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-16T00:11:00.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:11:00.195+0000 7f6aaa855640 -1 client.15234: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-16T00:11:00.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Exception in thread Thread-15: 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: Traceback (most recent call last): 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.run() 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.function(*self.args, **self.kwargs) 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: self.prune_snapshots(sched) 2024-09-16T00:11:00.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-16T00:11:00.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T00:11:00.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:00.428 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T00:11:00.428 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T00:11:00.428 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.b 2024-09-16T00:11:00.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:00 smithi081 systemd[1]: Stopping Ceph mon.b for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:01.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b[35056]: 2024-09-16T00:11:00.749+0000 7f0ff0478640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:01.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b[35056]: 2024-09-16T00:11:00.749+0000 7f0ff0478640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:11:01.057 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:00 smithi081 podman[69959]: 2024-09-16 00:11:00.973909695 +0000 UTC m=+0.346324781 container died 94c27af25f7a12143b63a1c2a63976b3067ac767870305bc4c315a98c8d9b804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:11:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 podman[69959]: 2024-09-16 00:11:01.416795785 +0000 UTC m=+0.789210909 container cleanup 94c27af25f7a12143b63a1c2a63976b3067ac767870305bc4c315a98c8d9b804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T00:11:01.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 bash[69959]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b 2024-09-16T00:11:01.893 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T00:11:01.916 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 podman[69973]: 2024-09-16 00:11:01.643393164 +0000 UTC m=+0.661187182 container remove 94c27af25f7a12143b63a1c2a63976b3067ac767870305bc4c315a98c8d9b804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:11:01.916 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-16T00:11:01.916 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 systemd[1]: Stopped Ceph mon.b for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-16T00:11:01.916 INFO:journalctl@ceph.mon.b.smithi081.stdout:Sep 16 00:11:01 smithi081 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.b.service: Consumed 12.205s CPU time. 2024-09-16T00:11:01.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:01.946 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T00:11:01.946 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T00:11:01.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.c 2024-09-16T00:11:02.265 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:01 smithi102 systemd[1]: Stopping Ceph mon.c for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:02.591 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c[34451]: 2024-09-16T00:11:02.263+0000 7f8ef83f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:02.591 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c[34451]: 2024-09-16T00:11:02.263+0000 7f8ef83f8640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:11:02.591 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:02 smithi102 podman[70054]: 2024-09-16 00:11:02.284398854 +0000 UTC m=+0.148495150 container died ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T00:11:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:02 smithi102 podman[70054]: 2024-09-16 00:11:02.689405689 +0000 UTC m=+0.553501984 container cleanup ae05a1caf9baf964e18b082b3eadac2ea67c3d8f37f0ef169d520d7cf41168d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:11:02.871 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 16 00:11:02 smithi102 bash[70054]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mon-c 2024-09-16T00:11:03.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T00:11:03.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:03.155 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T00:11:03.155 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T00:11:03.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:11:03.476 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 systemd[1]: Stopping Ceph mgr.x for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:03.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:11:03.475+0000 7f6b45a31640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:03.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x[31429]: 2024-09-16T00:11:03.475+0000 7f6b45a31640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T00:11:03.754 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 podman[89763]: 2024-09-16 00:11:03.506793967 +0000 UTC m=+0.146138046 container died b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:11:04.179 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 podman[89763]: 2024-09-16 00:11:03.91846646 +0000 UTC m=+0.557810525 container cleanup b8edf6bca87f32abae6eafaafaf34dc0f71ab00300be4ed8cc1d12183e8e399c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:11:04.179 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 16 00:11:03 smithi031 bash[89763]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-x 2024-09-16T00:11:04.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T00:11:04.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:04.407 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T00:11:04.407 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T00:11:04.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y 2024-09-16T00:11:04.743 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:04 smithi081 systemd[1]: Stopping Ceph mgr.y for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:05.056 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:04 smithi081 podman[70081]: 2024-09-16 00:11:04.742974728 +0000 UTC m=+0.143745245 container died 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T00:11:05.408 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 podman[70081]: 2024-09-16 00:11:05.181505851 +0000 UTC m=+0.582276399 container cleanup 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:05.408 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 bash[70081]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y 2024-09-16T00:11:05.653 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T00:11:05.675 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 podman[70095]: 2024-09-16 00:11:05.407990959 +0000 UTC m=+0.660156264 container remove 1577a96482f28138d7044dd7dcafcdb924ba7ed136d1b5dc152e5494759e653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:11:05.675 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-16T00:11:05.675 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-16T00:11:05.675 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 systemd[1]: Stopped Ceph mgr.y for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-16T00:11:05.675 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Sep 16 00:11:05 smithi081 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.y.service: Consumed 11.566s CPU time. 2024-09-16T00:11:05.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:05.688 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T00:11:05.689 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T00:11:05.689 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z 2024-09-16T00:11:06.032 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:05 smithi102 systemd[1]: Stopping Ceph mgr.z for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:06.354 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 podman[70176]: 2024-09-16 00:11:06.031942816 +0000 UTC m=+0.148336604 container died c74b0d77d51adfe329cee4ebaa0efaff40f623389879da05b361f0eed43781b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-16T00:11:06.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 podman[70176]: 2024-09-16 00:11:06.462067849 +0000 UTC m=+0.578461634 container cleanup c74b0d77d51adfe329cee4ebaa0efaff40f623389879da05b361f0eed43781b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:11:06.620 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 bash[70176]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z 2024-09-16T00:11:06.943 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T00:11:06.965 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 podman[70187]: 2024-09-16 00:11:06.714107204 +0000 UTC m=+0.671967167 container remove c74b0d77d51adfe329cee4ebaa0efaff40f623389879da05b361f0eed43781b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:11:06.966 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-16T00:11:06.966 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-16T00:11:06.966 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 systemd[1]: Stopped Ceph mgr.z for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-16T00:11:06.966 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Sep 16 00:11:06 smithi102 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.517s CPU time. 2024-09-16T00:11:06.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:06.983 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T00:11:06.983 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:11:06.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:11:07.308 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:07 smithi031 systemd[1]: Stopping Ceph osd.0 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:07.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-16T00:11:07.307+0000 7f00993ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:07.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-16T00:11:07.307+0000 7f00993ea640 -1 osd.0 94 *** Got signal Terminated *** 2024-09-16T00:11:07.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0[45238]: 2024-09-16T00:11:07.307+0000 7f00993ea640 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:13.004 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:12 smithi031 podman[89881]: 2024-09-16 00:11:12.750519179 +0000 UTC m=+5.556907386 container died 9410dbde6219e3d81ed60fcf47c52a0c4537a3b8cf6a82f4293305e138bee84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T00:11:13.384 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:13 smithi031 podman[89881]: 2024-09-16 00:11:13.182484761 +0000 UTC m=+5.988872968 container cleanup 9410dbde6219e3d81ed60fcf47c52a0c4537a3b8cf6a82f4293305e138bee84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:11:13.384 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:13 smithi031 bash[89881]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:11:13.754 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:13 smithi031 podman[89897]: 2024-09-16 00:11:13.383904513 +0000 UTC m=+0.631004252 container remove 9410dbde6219e3d81ed60fcf47c52a0c4537a3b8cf6a82f4293305e138bee84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T00:11:14.012 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:13 smithi031 podman[89982]: 2024-09-16 00:11:13.718772286 +0000 UTC m=+0.016881876 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:14.012 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:13 smithi031 podman[89982]: 2024-09-16 00:11:13.851674842 +0000 UTC m=+0.149784429 container create e90aab088af84c13080b0dc4f017d295eef58c3a3e447a3b7c86e2d7a5d6ec3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T00:11:14.504 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:14 smithi031 podman[89982]: 2024-09-16 00:11:14.121452136 +0000 UTC m=+0.419561723 container init e90aab088af84c13080b0dc4f017d295eef58c3a3e447a3b7c86e2d7a5d6ec3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T00:11:14.505 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:14 smithi031 podman[89982]: 2024-09-16 00:11:14.125203324 +0000 UTC m=+0.423312911 container start e90aab088af84c13080b0dc4f017d295eef58c3a3e447a3b7c86e2d7a5d6ec3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:11:14.505 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:14 smithi031 podman[89982]: 2024-09-16 00:11:14.2130654 +0000 UTC m=+0.511174994 container attach e90aab088af84c13080b0dc4f017d295eef58c3a3e447a3b7c86e2d7a5d6ec3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T00:11:14.505 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 00:11:14 smithi031 podman[89982]: 2024-09-16 00:11:14.325262514 +0000 UTC m=+0.623372111 container died e90aab088af84c13080b0dc4f017d295eef58c3a3e447a3b7c86e2d7a5d6ec3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:14.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:11:14.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:14.962 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:11:14.962 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:11:14.963 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:11:15.278 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:15 smithi081 systemd[1]: Stopping Ceph osd.1 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:15.556 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-16T00:11:15.277+0000 7fcf6fa69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:15.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-16T00:11:15.277+0000 7fcf6fa69640 -1 osd.1 94 *** Got signal Terminated *** 2024-09-16T00:11:15.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1[39532]: 2024-09-16T00:11:15.277+0000 7fcf6fa69640 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:20.929 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:20 smithi081 podman[70208]: 2024-09-16 00:11:20.618246905 +0000 UTC m=+5.465344935 container died d6e5dc3ff2f388761a8a308c44e1831256207e903bf12fcebb4ba3d8fc0fc383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:11:21.281 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:21 smithi081 podman[70208]: 2024-09-16 00:11:21.036777141 +0000 UTC m=+5.883875170 container cleanup d6e5dc3ff2f388761a8a308c44e1831256207e903bf12fcebb4ba3d8fc0fc383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:11:21.281 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:21 smithi081 bash[70208]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:11:21.556 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:21 smithi081 podman[70225]: 2024-09-16 00:11:21.280044389 +0000 UTC m=+0.658203788 container remove d6e5dc3ff2f388761a8a308c44e1831256207e903bf12fcebb4ba3d8fc0fc383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T00:11:22.042 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:21 smithi081 podman[70312]: 2024-09-16 00:11:21.599949034 +0000 UTC m=+0.015969610 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:22.042 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:21 smithi081 podman[70312]: 2024-09-16 00:11:21.739174207 +0000 UTC m=+0.155194773 container create 58922cf637c3d060481901699eb80d3e2822b20237a19fa9cadbba447691110c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:11:22.307 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:22 smithi081 podman[70312]: 2024-09-16 00:11:22.041916604 +0000 UTC m=+0.457937177 container init 58922cf637c3d060481901699eb80d3e2822b20237a19fa9cadbba447691110c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:11:22.307 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:22 smithi081 podman[70312]: 2024-09-16 00:11:22.045575859 +0000 UTC m=+0.461596431 container start 58922cf637c3d060481901699eb80d3e2822b20237a19fa9cadbba447691110c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T00:11:22.307 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:22 smithi081 podman[70312]: 2024-09-16 00:11:22.125717565 +0000 UTC m=+0.541738143 container attach 58922cf637c3d060481901699eb80d3e2822b20237a19fa9cadbba447691110c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T00:11:22.307 INFO:journalctl@ceph.osd.1.smithi081.stdout:Sep 16 00:11:22 smithi081 podman[70312]: 2024-09-16 00:11:22.248890168 +0000 UTC m=+0.664910733 container died 58922cf637c3d060481901699eb80d3e2822b20237a19fa9cadbba447691110c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:11:22.979 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:11:23.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:23.019 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:11:23.019 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:11:23.019 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:11:23.334 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:23 smithi102 systemd[1]: Stopping Ceph osd.2 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:23.620 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-16T00:11:23.332+0000 7fc11d449640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:23.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-16T00:11:23.332+0000 7fc11d449640 -1 osd.2 94 *** Got signal Terminated *** 2024-09-16T00:11:23.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2[39405]: 2024-09-16T00:11:23.332+0000 7fc11d449640 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:29.119 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:28 smithi102 podman[70302]: 2024-09-16 00:11:28.786732959 +0000 UTC m=+5.572154954 container died ffb9ca2a640b1c08c404af32db15dd9b90f33ca6bcf5eb09f9cd6e4344509ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:11:29.371 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:29 smithi102 podman[70302]: 2024-09-16 00:11:29.217528136 +0000 UTC m=+6.002950130 container cleanup ffb9ca2a640b1c08c404af32db15dd9b90f33ca6bcf5eb09f9cd6e4344509ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:11:29.371 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:29 smithi102 bash[70302]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:11:29.834 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:29 smithi102 podman[70317]: 2024-09-16 00:11:29.427187677 +0000 UTC m=+0.642339787 container remove ffb9ca2a640b1c08c404af32db15dd9b90f33ca6bcf5eb09f9cd6e4344509ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:11:30.120 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:29 smithi102 podman[70406]: 2024-09-16 00:11:29.737175404 +0000 UTC m=+0.015567741 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:30.121 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:29 smithi102 podman[70406]: 2024-09-16 00:11:29.870191353 +0000 UTC m=+0.148583723 container create 2dbb98a15b9f24ceacb2f81c2226c887901d2487d98585ebe70a97369328b9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:30.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:30 smithi102 podman[70406]: 2024-09-16 00:11:30.147830081 +0000 UTC m=+0.426222413 container init 2dbb98a15b9f24ceacb2f81c2226c887901d2487d98585ebe70a97369328b9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:30.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:30 smithi102 podman[70406]: 2024-09-16 00:11:30.151083433 +0000 UTC m=+0.429475761 container start 2dbb98a15b9f24ceacb2f81c2226c887901d2487d98585ebe70a97369328b9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:11:30.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:30 smithi102 podman[70406]: 2024-09-16 00:11:30.247976925 +0000 UTC m=+0.526369259 container attach 2dbb98a15b9f24ceacb2f81c2226c887901d2487d98585ebe70a97369328b9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:11:30.621 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 16 00:11:30 smithi102 podman[70406]: 2024-09-16 00:11:30.334438099 +0000 UTC m=+0.612830432 container died 2dbb98a15b9f24ceacb2f81c2226c887901d2487d98585ebe70a97369328b9c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T00:11:30.963 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:11:31.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:31.006 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:11:31.006 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:11:31.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:11:31.346 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:31 smithi031 systemd[1]: Stopping Ceph osd.3 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:31.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:31.343+0000 7fda0e3b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:31.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:31.343+0000 7fda0e3b7640 -1 osd.3 94 *** Got signal Terminated *** 2024-09-16T00:11:31.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:31 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:31.343+0000 7fda0e3b7640 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:33.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:32 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:32.816+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:33.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:33.551+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:34.068 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:33 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:33.807+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:34.504 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:34.067+0000 7fda069c5640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:09.024877+0000 front 2024-09-16T00:11:09.024900+0000 (oldest deadline 2024-09-16T00:11:33.125059+0000) 2024-09-16T00:11:34.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:34.555+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:35.036 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:34 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:34.783+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:35.504 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:35.035+0000 7fda069c5640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:09.024877+0000 front 2024-09-16T00:11:09.024900+0000 (oldest deadline 2024-09-16T00:11:33.125059+0000) 2024-09-16T00:11:35.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:35.513+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:35.871 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:35.753+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:36.254 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:36 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3[51699]: 2024-09-16T00:11:36.023+0000 7fda069c5640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:09.024877+0000 front 2024-09-16T00:11:09.024900+0000 (oldest deadline 2024-09-16T00:11:33.125059+0000) 2024-09-16T00:11:36.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:35 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:35.808+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:36.870 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:36.745+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:36.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:36.501+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:37.070 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:36 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:36.832+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:37.071 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:36 smithi031 podman[90100]: 2024-09-16 00:11:36.783307103 +0000 UTC m=+5.563585082 container died 56419a7ae595b2d67debee85e2098205aec199ebe3eec0daf8dfbc00e6bf0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:11:37.414 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:37 smithi031 podman[90100]: 2024-09-16 00:11:37.169854425 +0000 UTC m=+5.950132407 container cleanup 56419a7ae595b2d67debee85e2098205aec199ebe3eec0daf8dfbc00e6bf0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:11:37.414 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:37 smithi031 bash[90100]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:11:37.754 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:37 smithi031 podman[90117]: 2024-09-16 00:11:37.437777273 +0000 UTC m=+0.646395811 container remove 56419a7ae595b2d67debee85e2098205aec199ebe3eec0daf8dfbc00e6bf0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:11:37.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:37 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:37.412+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:37.767 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:37.476+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:38.084 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:37 smithi031 podman[90204]: 2024-09-16 00:11:37.707441192 +0000 UTC m=+0.016365515 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:38.084 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:37 smithi031 podman[90204]: 2024-09-16 00:11:37.830499834 +0000 UTC m=+0.139424146 container create 09025bc9d556132521f6a63a697beb6659f3b31733ce952da58939cf04e89269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:11:38.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:37 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:37.876+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:38.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:37.766+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:38.426 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:38 smithi031 podman[90204]: 2024-09-16 00:11:38.083885607 +0000 UTC m=+0.392809918 container init 09025bc9d556132521f6a63a697beb6659f3b31733ce952da58939cf04e89269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:11:38.426 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:38 smithi031 podman[90204]: 2024-09-16 00:11:38.087564242 +0000 UTC m=+0.396488559 container start 09025bc9d556132521f6a63a697beb6659f3b31733ce952da58939cf04e89269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:11:38.427 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:38 smithi031 podman[90204]: 2024-09-16 00:11:38.175392664 +0000 UTC m=+0.484316994 container attach 09025bc9d556132521f6a63a697beb6659f3b31733ce952da58939cf04e89269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T00:11:38.427 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 16 00:11:38 smithi031 podman[90204]: 2024-09-16 00:11:38.281108178 +0000 UTC m=+0.590032506 container died 09025bc9d556132521f6a63a697beb6659f3b31733ce952da58939cf04e89269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:11:38.746 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:38.442+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:38.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:38 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:38.424+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:38.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:38 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:38.366+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:38.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:11:39.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:39.014 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:11:39.014 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:11:39.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:11:39.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:38.746+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:39.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:38 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:38.847+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:39.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:39 smithi081 systemd[1]: Stopping Ceph osd.4 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:39.751 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:39.424+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:39.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:39 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:39.412+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:39.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:39 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:39.404+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:39.807 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:39 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-16T00:11:39.332+0000 7f8ea1f54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:39.807 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:39 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-16T00:11:39.332+0000 7f8ea1f54640 -1 osd.4 94 *** Got signal Terminated *** 2024-09-16T00:11:39.807 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:39 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-16T00:11:39.332+0000 7f8ea1f54640 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:40.124 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:39.750+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:40.345 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:39 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:39.853+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:40.708 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:40 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:40.396+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:40.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:40 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:40.443+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:40.806 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:40 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:40.408+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:41.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:40 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:40.707+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:41.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:40 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:40.809+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:41.438 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:41 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:41.146+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:41.620 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:41 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:41.354+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:41.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:41 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:41.354+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:41.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:41.412+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:41.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:41 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:41.437+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:41.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:41 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:41.437+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:42.069 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:41 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:41.709+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:42.069 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:41 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:41.709+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:42.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:41 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:41.846+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:42.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:42.068+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:42.452 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:42 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:42.098+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:42.693 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:42.383+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:42.694 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:42.383+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:42.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:42.395+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:42.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:42 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:42.451+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:42.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:42 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:42.451+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:43.071 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:42.693+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:43.071 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:42 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:42.693+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:43.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:42.848+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:43.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:43.070+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:43.494 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:43 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-16T00:11:43.199+0000 7f8e9e56a640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.117478+0000 front 2024-09-16T00:11:18.117414+0000 (oldest deadline 2024-09-16T00:11:42.817545+0000) 2024-09-16T00:11:43.495 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:43 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:43.105+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:43.717 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:43.413+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:43.717 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:43.413+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:43.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:43 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:43.425+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:43.806 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:43 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:43.493+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:43.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:43 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:43.493+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:44.116 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:43.716+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:44.116 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:43 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:43.716+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:44.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:43 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:43.897+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:44.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:44.115+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:44.542 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:44 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4[45935]: 2024-09-16T00:11:44.192+0000 7f8e9e56a640 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.117478+0000 front 2024-09-16T00:11:18.117414+0000 (oldest deadline 2024-09-16T00:11:42.817545+0000) 2024-09-16T00:11:44.543 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:44 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:44.096+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:44.748 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:44.434+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:44.748 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:44.434+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:44.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:44.447+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:44.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:44.447+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:44.807 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:44 smithi081 podman[70428]: 2024-09-16 00:11:44.665297213 +0000 UTC m=+5.448517743 container died b427dc2ddbcc05c95d1cc2472d9f3cc10d6e02dc8d2479978a16d01fe8b44bc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-16T00:11:44.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:44 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:44.541+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:44.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:44 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:44.541+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:45.117 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:44.748+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:45.117 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:44 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:44.748+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:45.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:44 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:44.900+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:45.307 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:45 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:45.142+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:45.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 podman[70428]: 2024-09-16 00:11:45.059010964 +0000 UTC m=+5.842231459 container cleanup b427dc2ddbcc05c95d1cc2472d9f3cc10d6e02dc8d2479978a16d01fe8b44bc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:11:45.308 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 bash[70428]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:11:45.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:45 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:45.116+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:45.685 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 podman[70443]: 2024-09-16 00:11:45.338794972 +0000 UTC m=+0.672792679 container remove b427dc2ddbcc05c95d1cc2472d9f3cc10d6e02dc8d2479978a16d01fe8b44bc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:11:45.685 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:45 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:45.556+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:45.685 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:45 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:45.556+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:45.753 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:45 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:45.483+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:45.753 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:45 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:45.483+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:45.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:45.490+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:45.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:45.490+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:45.997 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 podman[70533]: 2024-09-16 00:11:45.588025529 +0000 UTC m=+0.016299223 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:45.997 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 podman[70533]: 2024-09-16 00:11:45.719842126 +0000 UTC m=+0.148115865 container create 4496f9bbad3b64f99f111a71113fdbc75c99a8c59cd6e88fd0ede1c25be2bdf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:11:46.115 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:45 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:45.752+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:46.115 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:45 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:45.752+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:46.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:45.909+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:46.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:45.909+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:46.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:45 smithi081 podman[70533]: 2024-09-16 00:11:45.997369972 +0000 UTC m=+0.425643666 container init 4496f9bbad3b64f99f111a71113fdbc75c99a8c59cd6e88fd0ede1c25be2bdf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:11:46.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:46 smithi081 podman[70533]: 2024-09-16 00:11:46.000826045 +0000 UTC m=+0.429099753 container start 4496f9bbad3b64f99f111a71113fdbc75c99a8c59cd6e88fd0ede1c25be2bdf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:11:46.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:46 smithi081 podman[70533]: 2024-09-16 00:11:46.106115545 +0000 UTC m=+0.534389241 container attach 4496f9bbad3b64f99f111a71113fdbc75c99a8c59cd6e88fd0ede1c25be2bdf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:11:46.307 INFO:journalctl@ceph.osd.4.smithi081.stdout:Sep 16 00:11:46 smithi081 podman[70533]: 2024-09-16 00:11:46.189357968 +0000 UTC m=+0.617631662 container died 4496f9bbad3b64f99f111a71113fdbc75c99a8c59cd6e88fd0ede1c25be2bdf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:11:46.308 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:46 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:46.154+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:46.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:46.115+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:46.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:46 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:46.447+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:46.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:46 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:46.447+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:46.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:46 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:46.553+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:46.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:46 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:46.553+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:46.870 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:46.715+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:46.871 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:46.715+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:46.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:46.494+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:46.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:46 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:46.494+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:46.882 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:11:46.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:46.921 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:11:46.921 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:11:46.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:11:47.231 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:46 smithi102 systemd[1]: Stopping Ceph osd.5 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:47.231 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:47.152+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:47.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:46 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:46.871+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:47.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:46 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:46.871+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:47.507 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:47.230+0000 7f820480e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:47.507 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:47.230+0000 7f820480e640 -1 osd.5 94 *** Got signal Terminated *** 2024-09-16T00:11:47.508 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:47.230+0000 7f820480e640 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:47.556 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:47 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:47.142+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:47.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:47 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:47.428+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:47.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:47 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:47.428+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:47.870 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:47.682+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:47.870 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:47.683+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:47.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:47.506+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:47.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:47 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:47.506+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:48.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:47 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:47.577+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:48.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:47 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:47.577+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:48.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:47 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:47.912+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:48.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:47 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:47.912+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:48.472 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:48 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:48.189+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:48.543 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:48 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:48.183+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:48.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:48.415+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:48.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:48.415+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:48.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:48 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:48.542+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:48.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:48 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:48.542+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:48.873 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:48 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:48.717+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:48.873 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:48 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:48.717+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:48.873 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:48 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:48.472+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:48.873 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:48 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:48.472+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:49.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:48.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:49.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:48.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:49.450 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:49 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:49.186+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:49.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:49.149+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:49.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:49.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:49.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:49.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:49.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:49.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:49.763 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:49 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:49.449+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:49.763 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:49 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:49.449+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:50.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:49.584+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:50.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:49.584+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:50.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:49 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:49.584+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:50.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:49 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:49.763+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:50.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:49 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:49.763+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:50.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:49.860+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:50.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:49 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:49.860+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:50.479 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:50.193+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:50.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:50 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:50.151+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:50.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:50 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:50.151+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:50.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:50 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:50.461+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:50.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:50 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:50.461+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:50.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:50 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:50.461+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:50.779 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:50.478+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:50.779 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:50.478+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:50.779 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:50.478+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:51.056 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:50 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:50.609+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:51.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:50 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:50.609+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:51.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:50 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:50.609+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:51.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:50.778+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:51.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:50.778+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:51.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:50 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:50.778+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.679574+0000 front 2024-09-16T00:11:26.679555+0000 (oldest deadline 2024-09-16T00:11:50.179745+0000) 2024-09-16T00:11:51.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:50 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:50.848+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:51.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:50 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:50.848+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:51.516 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:51.229+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:51.516 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:51.229+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:51.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:51.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:51.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:51.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:51.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:51.425+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:51.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:51.425+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:51.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:51.425+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:51.793 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:51.515+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:51.793 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:51.515+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:51.793 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:51.515+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:52.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:51.570+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:52.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:51.570+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:52.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:51 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:51.570+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:52.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:51.792+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:07.277741+0000 front 2024-09-16T00:11:07.277727+0000 (oldest deadline 2024-09-16T00:11:35.478306+0000) 2024-09-16T00:11:52.120 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:51.792+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.478772+0000 front 2024-09-16T00:11:15.478877+0000 (oldest deadline 2024-09-16T00:11:41.378760+0000) 2024-09-16T00:11:52.121 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:51 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5[46275]: 2024-09-16T00:11:51.792+0000 7f8200e24640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.679574+0000 front 2024-09-16T00:11:26.679555+0000 (oldest deadline 2024-09-16T00:11:50.179745+0000) 2024-09-16T00:11:52.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:51.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:52.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:51 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:51.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:52.523 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:52 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:52.222+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:52.523 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:52 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:52.222+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:52.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:52 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:52.211+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:52.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:52 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:52.211+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:52.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:52.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:52.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:52.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:52.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:52.463+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:52.870 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:52 smithi102 podman[70520]: 2024-09-16 00:11:52.740695348 +0000 UTC m=+5.625207875 container died 123794bd807fc1f423d7f8a394e747181b119b2be36473ee0acf5cdbc3c7b708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:11:52.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:52 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:52.522+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:52.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:52 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:52.522+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:52.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:52 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:52.522+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:53.056 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:52 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:52.560+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:53.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:52 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:52.560+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:53.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:52 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:52.560+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:53.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:52.827+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:53.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:52 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:52.827+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:53.370 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70520]: 2024-09-16 00:11:53.138896014 +0000 UTC m=+6.023408543 container cleanup 123794bd807fc1f423d7f8a394e747181b119b2be36473ee0acf5cdbc3c7b708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:11:53.371 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 bash[70520]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:11:53.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:53 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:53.202+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:53.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:53 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:53.202+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:53.530 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:53 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:53.176+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:53.530 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:53 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:53.176+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:53.686 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70535]: 2024-09-16 00:11:53.38189358 +0000 UTC m=+0.643319086 container remove 123794bd807fc1f423d7f8a394e747181b119b2be36473ee0acf5cdbc3c7b708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:11:53.686 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:53 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:53.507+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:53.686 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:53 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:53.507+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:53.686 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:53 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:53.507+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:53.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:53.487+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:53.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:53.487+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:53.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:53.487+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:53.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:53 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:53.529+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:53.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:53 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:53.529+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:53.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:53 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:53.529+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:53.961 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70624]: 2024-09-16 00:11:53.588986147 +0000 UTC m=+0.016101372 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:11:53.961 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70624]: 2024-09-16 00:11:53.724645336 +0000 UTC m=+0.151760556 container create 905075def1834a865586f45fc40e70eb903dbae6f21f0f5bff872d6a8e6071ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:11:53.961 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70624]: 2024-09-16 00:11:53.960994898 +0000 UTC m=+0.388110117 container init 905075def1834a865586f45fc40e70eb903dbae6f21f0f5bff872d6a8e6071ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:11:54.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:53.791+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:54.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:53.791+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:54.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:53.791+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:54.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:54 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:54.175+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:54.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:54 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:54.175+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:54.371 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:53 smithi102 podman[70624]: 2024-09-16 00:11:53.964387065 +0000 UTC m=+0.391502286 container start 905075def1834a865586f45fc40e70eb903dbae6f21f0f5bff872d6a8e6071ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:11:54.371 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:54 smithi102 podman[70624]: 2024-09-16 00:11:54.035797078 +0000 UTC m=+0.462912298 container attach 905075def1834a865586f45fc40e70eb903dbae6f21f0f5bff872d6a8e6071ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T00:11:54.371 INFO:journalctl@ceph.osd.5.smithi102.stdout:Sep 16 00:11:54 smithi102 podman[70624]: 2024-09-16 00:11:54.15962271 +0000 UTC m=+0.586737934 container died 905075def1834a865586f45fc40e70eb903dbae6f21f0f5bff872d6a8e6071ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T00:11:54.541 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:54 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:54.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:54.541 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:54 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:54.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:54.748 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:54 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:54.480+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:54.748 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:54 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:54.480+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:54.748 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:54 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:54.480+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:54.769 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:11:54.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:11:54.807 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:11:54.808 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T00:11:54.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.6 2024-09-16T00:11:54.809 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:54 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:54.540+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:54.809 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:54 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:54.540+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:54.809 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:54 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:54.540+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:54.838 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:54.517+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:54.838 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:54.517+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:54.838 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:54.517+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:54.838 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:54.760+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:54.838 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:54.760+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:54.839 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:54 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:54.760+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:55.141 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:54 smithi031 systemd[1]: Stopping Ceph osd.6 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:11:55.463 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:55 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:55.201+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:55.463 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:55 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:55.201+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:55.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.140+0000 7fcd06ed6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:11:55.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.140+0000 7fcd06ed6640 -1 osd.6 94 *** Got signal Terminated *** 2024-09-16T00:11:55.504 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.140+0000 7fcd06ed6640 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:11:55.556 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:55 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:55.180+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:55.556 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:55 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:55.180+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:55.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:55 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:55.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:55.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:55 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:55.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:55.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:55 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:55.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:55.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:55 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:55.462+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:55.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:55 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:55.462+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:55.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:55 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:55.462+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:56.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.553+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:56.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.553+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:56.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:55.553+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:56.004 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:55.725+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:56.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:55.725+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:56.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:55 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:55.725+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:56.537 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:56.196+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:56.537 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:56.196+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:56.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:56 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:56.250+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:56.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:56 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:56.250+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:56.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:56 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:56.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:56.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:56 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:56.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:56.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:56 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:56.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:56.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:56.536+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:56.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:56.536+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:56.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:56.536+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:57.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:56.563+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:57.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:56.563+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:57.005 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:56.563+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:57.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:56.746+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:57.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:56.746+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:57.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:56 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:56.746+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:57.508 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:57.215+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:57.508 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:57.215+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:57.508 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:57.215+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:11:57.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:57 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:57.275+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:57.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:57 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:57.275+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:57.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:57 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:57.459+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:57.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:57 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:57.459+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:57.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:57 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:57.459+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:57.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:57.507+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:57.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:57.507+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:57.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:57 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:57.507+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:58.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:57.526+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:58.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:57.526+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:58.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:57.526+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:58.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:57.773+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:58.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:57.773+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:58.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:57 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:57.773+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:58.552 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:58.217+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:58.552 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:58.217+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:58.553 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:58.217+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:11:58.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:58.301+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:58.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:58.301+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:58.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:58.426+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:58.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:58.426+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:58.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:58 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:58.426+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:58.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:58.498+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:58.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:58.498+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:58.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:58.498+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:11:58.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:58.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:11:58.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:58.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:11:58.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:58 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:58.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:11:59.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:58.799+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:11:59.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:58.799+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:11:59.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:58.800+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:11:59.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:58 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:58.800+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:11:59.536 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:59 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:59.277+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:11:59.536 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:11:59 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:11:59.277+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:11:59.536 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:59 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:59.467+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:11:59.536 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:59 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:59.467+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:11:59.536 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:11:59 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:11:59.467+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:11:59.556 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:59.184+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:11:59.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:59.184+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:11:59.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:11:59.184+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:11:59.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:59.460+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.007011+0000 front 2024-09-16T00:11:11.007040+0000 (oldest deadline 2024-09-16T00:11:36.807833+0000) 2024-09-16T00:11:59.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:59.460+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.808524+0000 front 2024-09-16T00:11:16.808414+0000 (oldest deadline 2024-09-16T00:11:44.408959+0000) 2024-09-16T00:11:59.754 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6[58459]: 2024-09-16T00:11:59.460+0000 7fcd034ec640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.409564+0000 front 2024-09-16T00:11:24.409627+0000 (oldest deadline 2024-09-16T00:11:49.109472+0000) 2024-09-16T00:12:00.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:59.596+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:00.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:59.596+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:00.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:59.596+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:00.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:11:59 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:11:59.596+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:00.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:59.845+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:00.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:59.845+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:00.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:59.845+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:00.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:11:59 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:11:59.845+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:00.550 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:00.204+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:00.551 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:00.204+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:00.551 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:00.204+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:00.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:00.283+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:00.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:00.283+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:00.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:00.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:00.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:00.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:00.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:00 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:00.484+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:00.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:00.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:00.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:00.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:00.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:00.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:00.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:00.550+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:01.004 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:00 smithi031 podman[90320]: 2024-09-16 00:12:00.60652868 +0000 UTC m=+5.595774583 container died 468a9312ffa576cc1b70bd78deff666951bf2b2c2747a3fcfd74913bf84b014b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:12:01.004 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:00.828+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:01.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:00.828+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:01.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:00.828+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:01.005 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:00 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:00.828+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:01.375 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90320]: 2024-09-16 00:12:01.115530021 +0000 UTC m=+6.104775926 container cleanup 468a9312ffa576cc1b70bd78deff666951bf2b2c2747a3fcfd74913bf84b014b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:12:01.375 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 bash[90320]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6 2024-09-16T00:12:01.536 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:01.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:01.536 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:01.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:01.536 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:01.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:01.620 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:01.477+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:01.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:01.477+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:01.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:01.477+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:01.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:01.307+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:01.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:01.307+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:01.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:01 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:01.307+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:01.692 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90335]: 2024-09-16 00:12:01.375138307 +0000 UTC m=+0.757955633 container remove 468a9312ffa576cc1b70bd78deff666951bf2b2c2747a3fcfd74913bf84b014b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:12:01.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:01.535+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:01.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:01.535+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:01.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:01.535+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:01.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:01 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:01.535+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:01.953 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90426]: 2024-09-16 00:12:01.595333862 +0000 UTC m=+0.016436469 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:01.954 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90426]: 2024-09-16 00:12:01.717909524 +0000 UTC m=+0.139012163 container create 096c89840d25b4a876a799f869065af4a22f58a16428a58221acc503ad0b9c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:12:01.954 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:01 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:01.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:01.954 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:01 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:01.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:01.954 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:01 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:01.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:01.954 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:01 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:01.854+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:02.255 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90426]: 2024-09-16 00:12:01.953784078 +0000 UTC m=+0.374886684 container init 096c89840d25b4a876a799f869065af4a22f58a16428a58221acc503ad0b9c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:02.255 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:01 smithi031 podman[90426]: 2024-09-16 00:12:01.957478613 +0000 UTC m=+0.378581217 container start 096c89840d25b4a876a799f869065af4a22f58a16428a58221acc503ad0b9c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:12:02.255 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:02 smithi031 podman[90426]: 2024-09-16 00:12:02.086930537 +0000 UTC m=+0.508033146 container attach 096c89840d25b4a876a799f869065af4a22f58a16428a58221acc503ad0b9c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T00:12:02.255 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 16 00:12:02 smithi031 podman[90426]: 2024-09-16 00:12:02.160778487 +0000 UTC m=+0.581881087 container died 096c89840d25b4a876a799f869065af4a22f58a16428a58221acc503ad0b9c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:12:02.489 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:02.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:02.489 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:02.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:02.489 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:02.192+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:02.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:02.300+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:02.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:02.300+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:02.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:02.300+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:02.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:02.442+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:02.622 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:02.442+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:02.622 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:02 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:02.442+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:02.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:02.488+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:02.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:02.488+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:02.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:02.488+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:02.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:02 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:02.488+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:02.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T00:12:02.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:02.982 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T00:12:02.982 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T00:12:02.982 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.7 2024-09-16T00:12:03.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:02.883+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:03.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:02.883+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:03.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:02.883+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:03.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:02 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:02.883+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:03.296 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 systemd[1]: Stopping Ceph osd.7 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.208+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.208+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.208+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.295+0000 7fa615c8d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.295+0000 7fa615c8d640 -1 osd.7 94 *** Got signal Terminated *** 2024-09-16T00:12:03.297 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:03.295+0000 7fa615c8d640 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:12:03.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:03.495+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:03.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:03.495+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:03.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:03.495+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:03.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:03 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:03.495+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:03.620 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:03.320+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:03.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:03.320+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:03.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:03.320+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:03.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:03.473+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:03.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:03.473+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:03.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:03 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:03.473+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:04.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:03.914+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:04.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:03.914+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:04.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:03.914+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:04.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:03 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:03.914+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:04.468+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:04.468+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:04.468+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:04.468+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:04.220+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:04.220+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:04.557 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:04 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:04.220+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:04.870 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:04.370+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:04.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:04.370+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:04.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:04.370+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:04.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:04.504+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:04.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:04.504+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:04.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:04 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:04.504+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:05.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:04.933+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:05.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:04.933+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:05.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:04.933+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:05.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:04 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:04.933+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:05.428 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:05.175+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:05.428 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:05.175+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:05.428 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:05.175+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:05.620 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:05.533+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:05.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:05.533+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:05.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:05.533+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:05.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:05.345+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:05.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:05.345+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:05.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:05 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:05.345+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:05.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:05.426+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:05.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:05.426+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:05.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:05.426+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:05.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:05.426+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:06.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:05.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:06.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:05.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:06.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:05.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:06.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:05 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:05.891+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:06.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:06.158+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:06.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:06.158+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:06.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:06.158+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:06.575 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:06.321+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:06.576 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:06.321+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:06.576 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:06.321+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:06.576 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:06.321+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:06.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:06.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:06.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:06.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:06.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:06.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:06.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:06 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:06.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:06.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:06.574+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:06.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:06.574+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:06.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:06.574+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:06.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:06 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:06.574+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:07.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:06.896+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:07.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:06.896+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:07.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:06.896+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:07.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:06 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:06.896+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:07.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:07.156+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:07.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:07.156+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:07.474 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:07.156+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:07.595 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:07.341+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:07.595 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:07.341+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:07.595 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:07.341+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:07.595 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:07.341+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:07.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:07.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:07.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:07.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:07.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:07.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:07.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:07 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:07.473+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:07.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:07.593+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:07.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:07.593+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:07.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:07.593+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:07.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:07 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:07.593+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:08.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:07.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:08.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:07.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:08.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:07.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:08.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:07 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:07.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:08.520 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:08.108+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:16.413846+0000 front 2024-09-16T00:11:16.413800+0000 (oldest deadline 2024-09-16T00:11:41.113997+0000) 2024-09-16T00:12:08.520 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:08.108+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:23.915676+0000 front 2024-09-16T00:11:23.915546+0000 (oldest deadline 2024-09-16T00:11:49.815542+0000) 2024-09-16T00:12:08.521 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:08.108+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:32.116794+0000 front 2024-09-16T00:11:32.116847+0000 (oldest deadline 2024-09-16T00:11:56.816607+0000) 2024-09-16T00:12:08.521 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7[52728]: 2024-09-16T00:12:08.108+0000 7fa60e29b640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:42.717661+0000 front 2024-09-16T00:11:42.717633+0000 (oldest deadline 2024-09-16T00:12:07.417917+0000) 2024-09-16T00:12:08.620 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:08.582+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:08.582+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:08.582+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:08.582+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:08.367+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:08.367+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:08.367+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:08.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:08 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:08.367+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:08.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:08.520+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:08.980 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:08.520+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:08.980 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:08.520+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:08.981 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:08.520+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:08.981 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:08.520+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:09.148 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:08 smithi081 podman[70636]: 2024-09-16 00:12:08.819468665 +0000 UTC m=+5.646555750 container died b8a6333e40d5f0e6ece6513379cd55c5c2200eb6c21d4b041cae4f4b87cd5d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T00:12:09.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:08.927+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:09.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:08.927+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:09.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:08.928+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:09.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:08 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:08.928+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:09.483 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:09 smithi081 podman[70636]: 2024-09-16 00:12:09.247045224 +0000 UTC m=+6.074132300 container cleanup b8a6333e40d5f0e6ece6513379cd55c5c2200eb6c21d4b041cae4f4b87cd5d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:12:09.483 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:09 smithi081 bash[70636]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:09 smithi081 podman[70662]: 2024-09-16 00:12:09.4819042 +0000 UTC m=+0.663121377 container remove b8a6333e40d5f0e6ece6513379cd55c5c2200eb6c21d4b041cae4f4b87cd5d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:09 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:09.546+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:09 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:09.546+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:09 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:09.546+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:09 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:09.546+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:09.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:09 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:09.546+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:09.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:09.550+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:09.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:09.550+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:09.550+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:09.550+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:09.413+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:09.413+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:09.413+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:09.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:09 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:09.413+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:10.185 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:09 smithi081 podman[70751]: 2024-09-16 00:12:09.795797483 +0000 UTC m=+0.015724369 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:10.185 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:09 smithi081 podman[70751]: 2024-09-16 00:12:09.924459065 +0000 UTC m=+0.144385996 container create 00af52bd18ce83068a0709bf3ffa07238c9e1c5d9e20a93b25ae74b853f7298f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:12:10.185 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:10 smithi081 podman[70751]: 2024-09-16 00:12:10.185307183 +0000 UTC m=+0.405234069 container init 00af52bd18ce83068a0709bf3ffa07238c9e1c5d9e20a93b25ae74b853f7298f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T00:12:10.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:09.903+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:10.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:09.903+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:10.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:09.903+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:10.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:09.903+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:10.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:09 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:09.903+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:10.533 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:10 smithi081 podman[70751]: 2024-09-16 00:12:10.188770293 +0000 UTC m=+0.408697180 container start 00af52bd18ce83068a0709bf3ffa07238c9e1c5d9e20a93b25ae74b853f7298f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T00:12:10.534 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:10 smithi081 podman[70751]: 2024-09-16 00:12:10.260299382 +0000 UTC m=+0.480226281 container attach 00af52bd18ce83068a0709bf3ffa07238c9e1c5d9e20a93b25ae74b853f7298f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:12:10.534 INFO:journalctl@ceph.osd.7.smithi081.stdout:Sep 16 00:12:10 smithi081 podman[70751]: 2024-09-16 00:12:10.377118769 +0000 UTC m=+0.597045656 container died 00af52bd18ce83068a0709bf3ffa07238c9e1c5d9e20a93b25ae74b853f7298f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:12:10.580 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:10.460+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:10.580 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:10.460+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:10.580 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:10.460+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:10.580 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:10.460+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:10.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:10 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:10.533+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:10.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:10 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:10.533+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:10.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:10 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:10.533+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:10.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:10 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:10.533+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:10.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:10 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:10.533+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:10.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:10.579+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:10.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:10.579+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:10.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:10.579+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:10.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:10 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:10.579+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:11.060 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T00:12:11.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:11.101 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T00:12:11.101 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T00:12:11.102 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.8 2024-09-16T00:12:11.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:10.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:11.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:10.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:11.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:10.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:11.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:10.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:11.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:10 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:10.939+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:11.402 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 systemd[1]: Stopping Ceph osd.8 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:11.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:11.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:11.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:11.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:11.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.400+0000 7f08165c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.400+0000 7f08165c5640 -1 osd.8 94 *** Got signal Terminated *** 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.400+0000 7f08165c5640 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.464+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.464+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.464+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:11.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:11 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:11.464+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:12.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:11.557+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:12.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:11.557+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:12.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:11.557+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:12.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:11.557+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:12.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:11 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:11.557+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:12.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:11.959+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:12.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:11.959+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:12.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:11.959+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:12.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:11.959+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:12.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:11 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:11.959+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:12.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:12.477+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:12.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:12.477+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:12.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:12.477+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:12.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:12.477+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:12.872 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:12.662+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:12.872 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:12.662+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:12.872 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:12.662+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:12.872 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:12 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:12.662+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:13.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:12.586+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:13.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:12.586+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:13.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:12.586+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:13.058 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:12.586+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:13.058 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:12 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:12.586+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:13.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:12 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:12.967+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:13.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:12 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:12.967+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:13.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:12 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:12.967+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:13.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:12 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:12.967+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:13.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:12 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:12.967+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:13.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:13.654+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:13.654+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:13.654+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:13.654+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:13.446+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:13.446+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:13.446+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:13.446+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:13.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:13.446+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:50.550822+0000 front 2024-09-16T00:11:50.550881+0000 (oldest deadline 2024-09-16T00:12:12.850955+0000) 2024-09-16T00:12:14.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:13.620+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:14.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:13.620+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:14.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:13.620+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:14.058 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:13.620+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:14.058 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:13 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:13.620+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:14.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:14.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:14.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:14.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:14.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:14.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:14.680+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:14.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:14.680+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:14.680+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:14.680+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:14.451+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:14.451+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:14.451+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:14.451+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:14.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:14 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:14.451+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:50.550822+0000 front 2024-09-16T00:11:50.550881+0000 (oldest deadline 2024-09-16T00:12:12.850955+0000) 2024-09-16T00:12:15.040 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:15.040 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:15.041 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:15.041 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:15.041 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:15.041 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:14 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:14.578+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:15.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.962+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:15.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.962+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:15.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.962+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:15.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.962+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:15.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:14 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:14.962+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:15.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:15.636+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:15.636+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:15.636+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:15.636+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:15.636+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:15.409+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:15.409+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:15.409+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:15.409+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:15.871 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:15 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:15.409+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:50.550822+0000 front 2024-09-16T00:11:50.550881+0000 (oldest deadline 2024-09-16T00:12:12.850955+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:16.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:15 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:15.614+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:16.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:15 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:15.994+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:16.361+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:15.144731+0000 front 2024-09-16T00:11:15.144667+0000 (oldest deadline 2024-09-16T00:11:41.545072+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:16.362+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.845730+0000 front 2024-09-16T00:11:26.845759+0000 (oldest deadline 2024-09-16T00:11:50.945876+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:16.362+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:35.548070+0000 front 2024-09-16T00:11:35.547965+0000 (oldest deadline 2024-09-16T00:12:00.847745+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:16.362+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.149793+0000 front 2024-09-16T00:11:44.149843+0000 (oldest deadline 2024-09-16T00:12:05.849773+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8[53012]: 2024-09-16T00:12:16.362+0000 7f0812bdb640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:50.550822+0000 front 2024-09-16T00:11:50.550881+0000 (oldest deadline 2024-09-16T00:12:12.850955+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:16.608+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:16.608+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:16.608+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:16.621 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:16.608+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:16.622 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:16.608+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:17.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:16 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:16.635+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:17.060 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:16 smithi102 podman[70741]: 2024-09-16 00:12:16.761052251 +0000 UTC m=+5.472469921 container died 0ff57c166170582b89f0b8d2121664224644ace4cf28224ff1c8d2da08025f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:12:17.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:17.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:17.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:17.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:17.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:17.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:16 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:16.981+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:17.370 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70741]: 2024-09-16 00:12:17.159475883 +0000 UTC m=+5.870893552 container cleanup 0ff57c166170582b89f0b8d2121664224644ace4cf28224ff1c8d2da08025f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T00:12:17.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 bash[70741]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8 2024-09-16T00:12:17.733 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70756]: 2024-09-16 00:12:17.42761971 +0000 UTC m=+0.663051560 container remove 0ff57c166170582b89f0b8d2121664224644ace4cf28224ff1c8d2da08025f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:12:17.733 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:17.625+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:17.733 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:17.625+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:17.734 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:17.625+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:17.734 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:17.625+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:17.734 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:17 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:17.625+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:17.989 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70843]: 2024-09-16 00:12:17.636015427 +0000 UTC m=+0.015876916 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:17.989 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70843]: 2024-09-16 00:12:17.756518885 +0000 UTC m=+0.136380367 container create 9277a8335b75e48aa955cba63edde8ac20c50e5c72bd36c2825f7783cd455981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T00:12:17.989 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70843]: 2024-09-16 00:12:17.989283576 +0000 UTC m=+0.369145059 container init 9277a8335b75e48aa955cba63edde8ac20c50e5c72bd36c2825f7783cd455981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:18.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:17 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:17.594+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:18.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:17 smithi102 podman[70843]: 2024-09-16 00:12:17.992722686 +0000 UTC m=+0.372584169 container start 9277a8335b75e48aa955cba63edde8ac20c50e5c72bd36c2825f7783cd455981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:12:18.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:18 smithi102 podman[70843]: 2024-09-16 00:12:18.072551299 +0000 UTC m=+0.452412796 container attach 9277a8335b75e48aa955cba63edde8ac20c50e5c72bd36c2825f7783cd455981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:18.371 INFO:journalctl@ceph.osd.8.smithi102.stdout:Sep 16 00:12:18 smithi102 podman[70843]: 2024-09-16 00:12:18.187798061 +0000 UTC m=+0.567659545 container died 9277a8335b75e48aa955cba63edde8ac20c50e5c72bd36c2825f7783cd455981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:18.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:18.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:18.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:18.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:18.505 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:18.505 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:18.023+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:18.785 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:18 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:18.583+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:18.785 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:18 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:18.583+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:18.785 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:18 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:18.583+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:18.785 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:18 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:18.583+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:18.785 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:18 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:18.583+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:18.803 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T00:12:18.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:18.843 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T00:12:18.843 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T00:12:18.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.9 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:19.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:18 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:18.587+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:18 smithi031 systemd[1]: Stopping Ceph osd.9 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:19.169 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:19.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.168+0000 7f53b02c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:19.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.168+0000 7f53b02c1640 -1 osd.9 94 *** Got signal Terminated *** 2024-09-16T00:12:19.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:19 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:19.168+0000 7f53b02c1640 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:12:19.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:19 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:19.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:19.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:19 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:19.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:19.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:19 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:19.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:19.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:19 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:19.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:19.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:19 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:19.618+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:20.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:19 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:19.583+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:20.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:20.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:20.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:20.505 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:20.505 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:20.505 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.028+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:21.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:20 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:20.573+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:21.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:20 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:20.621+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:21.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:20 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:20.621+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:21.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:20 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:20.621+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:21.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:20 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:20.621+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:21.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:20 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:20.621+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:21.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:20 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:20.988+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:21.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:21.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:21.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:21.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:21.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:21.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:21.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:21 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:21.547+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:21.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:21.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:21.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:21.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:21.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:21.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:21.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:21.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:21.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:21.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:22.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:22.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:22.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:22.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:22.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:22.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.000+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:22.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:22.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:22.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:22.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:22.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:22.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:22.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:22 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:22.505+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:22.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:22 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:22.605+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:23.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:23.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:23.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:23.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:23.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:23.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:22 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:22.960+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:23.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:23 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:23.479+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:23.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:23.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:23.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:23.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:23.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:23.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:23 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:23.613+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:24.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:08.072722+0000 front 2024-09-16T00:11:08.072450+0000 (oldest deadline 2024-09-16T00:11:32.772668+0000) 2024-09-16T00:12:24.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:19.774016+0000 front 2024-09-16T00:11:19.774195+0000 (oldest deadline 2024-09-16T00:11:45.674150+0000) 2024-09-16T00:12:24.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:26.175251+0000 front 2024-09-16T00:11:26.175166+0000 (oldest deadline 2024-09-16T00:11:53.176090+0000) 2024-09-16T00:12:24.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:33.676953+0000 front 2024-09-16T00:11:33.676875+0000 (oldest deadline 2024-09-16T00:11:58.377063+0000) 2024-09-16T00:12:24.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:44.278175+0000 front 2024-09-16T00:11:44.278266+0000 (oldest deadline 2024-09-16T00:12:09.577886+0000) 2024-09-16T00:12:24.255 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:23 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9[65433]: 2024-09-16T00:12:23.958+0000 7f53ac8d7640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.578421+0000 front 2024-09-16T00:11:49.578551+0000 (oldest deadline 2024-09-16T00:12:15.478280+0000) 2024-09-16T00:12:24.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:24.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:24.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:24.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:24.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:24.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:24.808 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:24 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:24.489+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:24.818 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:24 smithi031 podman[90541]: 2024-09-16 00:12:24.561498979 +0000 UTC m=+5.516489835 container died 8552a8aeb9a1668164ca321d3190ba01fa12d63a635465fc851cc48dcc7ee2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T00:12:24.870 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:24.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:24.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:24.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:24.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:24.871 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:24.585+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:25.203 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:24 smithi031 podman[90541]: 2024-09-16 00:12:24.925761366 +0000 UTC m=+5.880752277 container cleanup 8552a8aeb9a1668164ca321d3190ba01fa12d63a635465fc851cc48dcc7ee2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:12:25.203 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:24 smithi031 bash[90541]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9 2024-09-16T00:12:25.504 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90556]: 2024-09-16 00:12:25.201860977 +0000 UTC m=+0.634117823 container remove 8552a8aeb9a1668164ca321d3190ba01fa12d63a635465fc851cc48dcc7ee2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:25.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:25 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:25.444+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:25.855 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.454043163 +0000 UTC m=+0.016598683 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:25.855 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.560921207 +0000 UTC m=+0.123476756 container create 157420d2bca0e19394f54d224faa57427d0d035bcba8c1671e7e8a6b0ce5a024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:12:25.855 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.788333451 +0000 UTC m=+0.350888965 container init 157420d2bca0e19394f54d224faa57427d0d035bcba8c1671e7e8a6b0ce5a024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T00:12:25.855 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.792163346 +0000 UTC m=+0.354718857 container start 157420d2bca0e19394f54d224faa57427d0d035bcba8c1671e7e8a6b0ce5a024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:25.855 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.854644986 +0000 UTC m=+0.417200504 container attach 157420d2bca0e19394f54d224faa57427d0d035bcba8c1671e7e8a6b0ce5a024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T00:12:26.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:26.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:26.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:26.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:26.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:26.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:25 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:25.628+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:26.254 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 16 00:12:25 smithi031 podman[90644]: 2024-09-16 00:12:25.988837826 +0000 UTC m=+0.551393343 container died 157420d2bca0e19394f54d224faa57427d0d035bcba8c1671e7e8a6b0ce5a024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:12:26.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T00:12:26.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:26.599 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T00:12:26.599 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T00:12:26.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.10 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.441+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:26.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 systemd[1]: Stopping Ceph osd.10 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:27.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:26 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:26.675+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:27.306 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.908+0000 7ff8c9fff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:27.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.908+0000 7ff8c9fff640 -1 osd.10 94 *** Got signal Terminated *** 2024-09-16T00:12:27.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:26 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:26.908+0000 7ff8c9fff640 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:27.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:27 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:27.485+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:28.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:28.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:28.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:28.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:28.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:28.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:27 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:27.658+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:28.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:28 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:28.492+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:29.022 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:28 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:28.706+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:29.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:29 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:29.519+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:30.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.720+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:30.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.720+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:30.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.720+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:30.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.721+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:30.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.721+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:30.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:29 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:29.721+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:30.807 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:30 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:30.555+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:31.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:31.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:31.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:31.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:31.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:31.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:30 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:30.723+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:11.987038+0000 front 2024-09-16T00:11:11.987234+0000 (oldest deadline 2024-09-16T00:11:37.887041+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.388138+0000 front 2024-09-16T00:11:18.388185+0000 (oldest deadline 2024-09-16T00:11:40.688413+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.590412+0000 front 2024-09-16T00:11:24.590296+0000 (oldest deadline 2024-09-16T00:11:49.290400+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-16T00:11:36.292099+0000 front 2024-09-16T00:11:36.292059+0000 (oldest deadline 2024-09-16T00:11:59.192062+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:39.192522+0000 front 2024-09-16T00:11:39.192493+0000 (oldest deadline 2024-09-16T00:12:07.993179+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:47.993922+0000 front 2024-09-16T00:11:47.993769+0000 (oldest deadline 2024-09-16T00:12:13.893691+0000) 2024-09-16T00:12:32.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:31 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10[59254]: 2024-09-16T00:12:31.576+0000 7ff8c6615640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:57.994868+0000 front 2024-09-16T00:11:57.994791+0000 (oldest deadline 2024-09-16T00:12:21.494719+0000) 2024-09-16T00:12:32.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:32.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:31 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:31.769+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:32.667 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:32 smithi081 podman[70856]: 2024-09-16 00:12:32.39047601 +0000 UTC m=+5.599254048 container died 0e674e27bc89506468816cd9c5fc8b49ac00fb9bec4611df7648f2240bc0fc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:12:33.039 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:32 smithi081 podman[70856]: 2024-09-16 00:12:32.783628729 +0000 UTC m=+5.992406766 container cleanup 0e674e27bc89506468816cd9c5fc8b49ac00fb9bec4611df7648f2240bc0fc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:33.039 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:32 smithi081 bash[70856]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10 2024-09-16T00:12:33.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:33.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:32 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:32.785+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:33.307 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70873]: 2024-09-16 00:12:33.039062515 +0000 UTC m=+0.647232556 container remove 0e674e27bc89506468816cd9c5fc8b49ac00fb9bec4611df7648f2240bc0fc0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:12:33.713 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.291054232 +0000 UTC m=+0.016101673 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:33.713 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.419241164 +0000 UTC m=+0.144288592 container create def99f27b3f1553445433112ae550e8e4499c700bef202800f9d397579821721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:12:33.713 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.629788497 +0000 UTC m=+0.354835925 container init def99f27b3f1553445433112ae550e8e4499c700bef202800f9d397579821721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:12:33.713 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.632928058 +0000 UTC m=+0.357975487 container start def99f27b3f1553445433112ae550e8e4499c700bef202800f9d397579821721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:12:33.713 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.713110034 +0000 UTC m=+0.438157465 container attach def99f27b3f1553445433112ae550e8e4499c700bef202800f9d397579821721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T00:12:34.057 INFO:journalctl@ceph.osd.10.smithi081.stdout:Sep 16 00:12:33 smithi081 podman[70961]: 2024-09-16 00:12:33.836163557 +0000 UTC m=+0.561211001 container died def99f27b3f1553445433112ae550e8e4499c700bef202800f9d397579821721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-16T00:12:34.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:34.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:33 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:33.797+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:34.427 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T00:12:34.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:34.465 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T00:12:34.466 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T00:12:34.466 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.11 2024-09-16T00:12:34.779 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 systemd[1]: Stopping Ceph osd.11 for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.778+0000 7fbd58475640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.778+0000 7fbd58475640 -1 osd.11 94 *** Got signal Terminated *** 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.778+0000 7fbd58475640 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:35.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:34 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:34.786+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:36.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:36.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:35 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:35.768+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:37.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:36 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:36.733+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:38.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:37 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:37.752+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-09-16T00:12:13.329810+0000 front 2024-09-16T00:12:13.329828+0000 (oldest deadline 2024-09-16T00:12:37.429854+0000) 2024-09-16T00:12:39.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:39.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:38 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:38.777+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-09-16T00:12:13.329810+0000 front 2024-09-16T00:12:13.329828+0000 (oldest deadline 2024-09-16T00:12:37.429854+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T00:11:10.019763+0000 front 2024-09-16T00:11:10.019749+0000 (oldest deadline 2024-09-16T00:11:33.519849+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-09-16T00:11:18.220949+0000 front 2024-09-16T00:11:18.220925+0000 (oldest deadline 2024-09-16T00:11:41.120971+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-09-16T00:11:24.522175+0000 front 2024-09-16T00:11:24.522207+0000 (oldest deadline 2024-09-16T00:11:49.822352+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-09-16T00:11:43.324592+0000 front 2024-09-16T00:11:43.324610+0000 (oldest deadline 2024-09-16T00:12:05.624578+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-09-16T00:11:49.725482+0000 front 2024-09-16T00:11:49.725420+0000 (oldest deadline 2024-09-16T00:12:15.025549+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-16T00:11:59.527376+0000 front 2024-09-16T00:11:59.527454+0000 (oldest deadline 2024-09-16T00:12:21.827330+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-09-16T00:12:06.428907+0000 front 2024-09-16T00:12:06.428849+0000 (oldest deadline 2024-09-16T00:12:31.029040+0000) 2024-09-16T00:12:40.121 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:39 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11[59666]: 2024-09-16T00:12:39.758+0000 7fbd54a8b640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.102:6820 osd.8 since back 2024-09-16T00:12:13.329810+0000 front 2024-09-16T00:12:13.329828+0000 (oldest deadline 2024-09-16T00:12:37.429854+0000) 2024-09-16T00:12:40.430 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:40 smithi102 podman[70960]: 2024-09-16 00:12:40.173347649 +0000 UTC m=+5.514233436 container died 3916da660a8eff3303666e0a2822d2e14f1ed0d5304d1a0fa6b7a7149f366b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T00:12:40.814 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:40 smithi102 podman[70960]: 2024-09-16 00:12:40.520812561 +0000 UTC m=+5.861698346 container cleanup 3916da660a8eff3303666e0a2822d2e14f1ed0d5304d1a0fa6b7a7149f366b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:12:40.814 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:40 smithi102 bash[70960]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11 2024-09-16T00:12:41.120 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:40 smithi102 podman[70976]: 2024-09-16 00:12:40.813453442 +0000 UTC m=+0.640385634 container remove 3916da660a8eff3303666e0a2822d2e14f1ed0d5304d1a0fa6b7a7149f366b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:12:41.425 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.029854256 +0000 UTC m=+0.015476002 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:12:41.425 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.139100355 +0000 UTC m=+0.124722101 container create 39e8e7c8c101c7a5275fc6c5cac62772004e23f3d70d66c59ce99e7dfbf97e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:12:41.426 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.349909845 +0000 UTC m=+0.335531588 container init 39e8e7c8c101c7a5275fc6c5cac62772004e23f3d70d66c59ce99e7dfbf97e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T00:12:41.426 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.35316304 +0000 UTC m=+0.338784786 container start 39e8e7c8c101c7a5275fc6c5cac62772004e23f3d70d66c59ce99e7dfbf97e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T00:12:41.426 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.424670831 +0000 UTC m=+0.410292585 container attach 39e8e7c8c101c7a5275fc6c5cac62772004e23f3d70d66c59ce99e7dfbf97e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:12:41.827 INFO:journalctl@ceph.osd.11.smithi102.stdout:Sep 16 00:12:41 smithi102 podman[71065]: 2024-09-16 00:12:41.537367715 +0000 UTC m=+0.522989459 container died 39e8e7c8c101c7a5275fc6c5cac62772004e23f3d70d66c59ce99e7dfbf97e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:12:42.156 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T00:12:42.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:42.198 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T00:12:42.198 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T00:12:42.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.a 2024-09-16T00:12:42.477 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:42 smithi031 systemd[1]: Stopping Ceph mds.a for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:42.754 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: 2024-09-16T00:12:42.475+0000 7f103dba8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:42.754 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:42 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a[72620]: 2024-09-16T00:12:42.475+0000 7f103dba8640 -1 mds.a *** got signal Terminated *** 2024-09-16T00:12:44.119 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:43 smithi031 podman[90752]: 2024-09-16 00:12:43.851856858 +0000 UTC m=+1.452744702 container died 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:12:44.486 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:44 smithi031 podman[90752]: 2024-09-16 00:12:44.226274693 +0000 UTC m=+1.827162541 container cleanup 6f89ad31d670377dc25d68a840f4057986d3b83d49d915d2108682249d202469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:12:44.486 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 16 00:12:44 smithi031 bash[90752]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-a 2024-09-16T00:12:44.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T00:12:44.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:44.695 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T00:12:44.695 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T00:12:44.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.d 2024-09-16T00:12:45.030 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:44 smithi031 systemd[1]: Stopping Ceph mds.d for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:45.504 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d[71414]: 2024-09-16T00:12:45.030+0000 7f400c9aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:45.504 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:45 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d[71414]: 2024-09-16T00:12:45.030+0000 7f400c9aa640 -1 mds.d *** got signal Terminated *** 2024-09-16T00:12:47.254 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:46 smithi031 podman[90875]: 2024-09-16 00:12:46.980042702 +0000 UTC m=+2.060348435 container died f61cf5472edb947dfcf7390cbc89b7ea1f99d6ca6c00f539229911448f2dffd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:12:47.603 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:47 smithi031 podman[90875]: 2024-09-16 00:12:47.279515922 +0000 UTC m=+2.359821647 container cleanup f61cf5472edb947dfcf7390cbc89b7ea1f99d6ca6c00f539229911448f2dffd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:12:47.604 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 16 00:12:47 smithi031 bash[90875]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-d 2024-09-16T00:12:47.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T00:12:47.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:47.811 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T00:12:47.811 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T00:12:47.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.g 2024-09-16T00:12:48.151 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:47 smithi031 systemd[1]: Stopping Ceph mds.g for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:48.504 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-g[72287]: 2024-09-16T00:12:48.151+0000 7f18b5b2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:48.504 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:48 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-g[72287]: 2024-09-16T00:12:48.151+0000 7f18b5b2d640 -1 mds.g *** got signal Terminated *** 2024-09-16T00:12:52.118 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:51 smithi031 podman[91007]: 2024-09-16 00:12:51.848112342 +0000 UTC m=+3.806995193 container died 6b9e423ae81be97a383aeb856837627da4872da7c95122c0f7b4b53ebb066b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:52.476 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:52 smithi031 podman[91007]: 2024-09-16 00:12:52.200180531 +0000 UTC m=+4.159063382 container cleanup 6b9e423ae81be97a383aeb856837627da4872da7c95122c0f7b4b53ebb066b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T00:12:52.476 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 16 00:12:52 smithi031 bash[91007]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-g 2024-09-16T00:12:52.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T00:12:52.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:52.675 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T00:12:52.675 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T00:12:52.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.j 2024-09-16T00:12:53.004 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:52 smithi031 systemd[1]: Stopping Ceph mds.j for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:53.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j[71016]: 2024-09-16T00:12:53.012+0000 7fc619006640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:53.504 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:53 smithi031 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j[71016]: 2024-09-16T00:12:53.012+0000 7fc619006640 -1 mds.j *** got signal Terminated *** 2024-09-16T00:12:55.621 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:55 smithi031 podman[91126]: 2024-09-16 00:12:55.361409816 +0000 UTC m=+2.463920817 container died cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:12:55.972 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:55 smithi031 podman[91126]: 2024-09-16 00:12:55.729048058 +0000 UTC m=+2.831559067 container cleanup cc669271f99bb2ca739638ee74d2c4cb8158e926110b3af818343cb3a8796fa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:12:55.972 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 16 00:12:55 smithi031 bash[91126]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-j 2024-09-16T00:12:56.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T00:12:56.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:12:56.141 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T00:12:56.141 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T00:12:56.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.b 2024-09-16T00:12:56.528 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:12:56 smithi081 systemd[1]: Stopping Ceph mds.b for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:12:56.806 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:12:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-16T00:12:56.527+0000 7f4707751640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:12:56.806 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:12:56 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b[64729]: 2024-09-16T00:12:56.527+0000 7f4707751640 -1 mds.b *** got signal Terminated *** 2024-09-16T00:13:00.177 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:12:59 smithi081 podman[71076]: 2024-09-16 00:12:59.854525693 +0000 UTC m=+3.517438167 container died 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:13:00.177 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:13:00 smithi081 podman[71076]: 2024-09-16 00:13:00.176268616 +0000 UTC m=+3.839181098 container cleanup 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:13:00.556 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:13:00 smithi081 bash[71076]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b 2024-09-16T00:13:00.557 INFO:journalctl@ceph.mds.b.smithi081.stdout:Sep 16 00:13:00 smithi081 podman[71088]: 2024-09-16 00:13:00.385732608 +0000 UTC m=+0.523524840 container remove 4fd20b705047719b5aedf94d820560f05a20f3fdf4c3055733ad934a7e11acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:13:00.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T00:13:00.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:00.654 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T00:13:00.654 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T00:13:00.654 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.e 2024-09-16T00:13:00.972 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:00 smithi081 systemd[1]: Stopping Ceph mds.e for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:01.306 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: 2024-09-16T00:13:00.971+0000 7fee2d16f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:01.307 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:00 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e[64110]: 2024-09-16T00:13:00.971+0000 7fee2d16f640 -1 mds.e *** got signal Terminated *** 2024-09-16T00:13:04.239 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:03 smithi081 podman[71197]: 2024-09-16 00:13:03.859394612 +0000 UTC m=+2.988051796 container died 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T00:13:04.502 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:04 smithi081 podman[71197]: 2024-09-16 00:13:04.355159248 +0000 UTC m=+3.483816431 container cleanup 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T00:13:04.502 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:04 smithi081 bash[71197]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e 2024-09-16T00:13:04.755 INFO:journalctl@ceph.mds.e.smithi081.stdout:Sep 16 00:13:04 smithi081 podman[71208]: 2024-09-16 00:13:04.606618528 +0000 UTC m=+0.746147105 container remove 3bb5c4de5f439c5869e3488cd72f8de9bcaa758856189a7cb24437a2c4363525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:13:04.759 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T00:13:04.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:04.793 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T00:13:04.793 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T00:13:04.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.h 2024-09-16T00:13:05.056 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:04 smithi081 systemd[1]: Stopping Ceph mds.h for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:05.557 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: 2024-09-16T00:13:05.192+0000 7f4c91f17640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:05.557 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:05 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h[64411]: 2024-09-16T00:13:05.192+0000 7f4c91f17640 -1 mds.h *** got signal Terminated *** 2024-09-16T00:13:08.136 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:07 smithi081 podman[71329]: 2024-09-16 00:13:07.851820243 +0000 UTC m=+2.815984508 container died fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:13:08.477 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:08 smithi081 podman[71329]: 2024-09-16 00:13:08.234426719 +0000 UTC m=+3.198590972 container cleanup fa20050466edafab8597679058648698c9c12ecfb8fc8370ac7fa4006fa6c4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T00:13:08.477 INFO:journalctl@ceph.mds.h.smithi081.stdout:Sep 16 00:13:08 smithi081 bash[71329]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-h 2024-09-16T00:13:08.611 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T00:13:08.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:08.646 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T00:13:08.646 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T00:13:08.646 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.k 2024-09-16T00:13:08.806 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:08 smithi081 systemd[1]: Stopping Ceph mds.k for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:09.306 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: 2024-09-16T00:13:08.979+0000 7feb0b315640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:09.307 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:08 smithi081 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k[63812]: 2024-09-16T00:13:08.979+0000 7feb0b315640 -1 mds.k *** got signal Terminated *** 2024-09-16T00:13:12.138 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:11 smithi081 podman[71450]: 2024-09-16 00:13:11.85734716 +0000 UTC m=+2.994027611 container died ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:13:12.481 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:12 smithi081 podman[71450]: 2024-09-16 00:13:12.229901428 +0000 UTC m=+3.366581878 container cleanup ad4fb911080be92e0af762e2abaff0d5d4ba2a9bdbe9e53b0d624812cbee5b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:13:12.481 INFO:journalctl@ceph.mds.k.smithi081.stdout:Sep 16 00:13:12 smithi081 bash[71450]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-k 2024-09-16T00:13:12.610 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T00:13:12.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:12.643 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T00:13:12.643 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T00:13:12.643 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.c 2024-09-16T00:13:13.021 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:12 smithi102 systemd[1]: Stopping Ceph mds.c for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:13.278 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: 2024-09-16T00:13:13.020+0000 7f0682dbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:13.278 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:13 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c[64462]: 2024-09-16T00:13:13.020+0000 7f0682dbd640 -1 mds.c *** got signal Terminated *** 2024-09-16T00:13:16.112 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:15 smithi102 podman[71168]: 2024-09-16 00:13:15.85308936 +0000 UTC m=+3.008573401 container died 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T00:13:16.371 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 podman[71168]: 2024-09-16 00:13:16.203416885 +0000 UTC m=+3.358900927 container cleanup 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:13:16.371 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 bash[71168]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c 2024-09-16T00:13:16.623 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T00:13:16.645 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 podman[71194]: 2024-09-16 00:13:16.379539768 +0000 UTC m=+0.517737813 container remove 033e62d3d88d0f5ff67aed716b868d9f0a3c7c0eb6c600d6c26d8475d459615f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:13:16.645 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-16T00:13:16.645 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 systemd[1]: Stopped Ceph mds.c for 21a99a0a-73bd-11ef-bceb-c7b262605968. 2024-09-16T00:13:16.646 INFO:journalctl@ceph.mds.c.smithi102.stdout:Sep 16 00:13:16 smithi102 systemd[1]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.c.service: Consumed 1.824s CPU time. 2024-09-16T00:13:16.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:16.659 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T00:13:16.659 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T00:13:16.659 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.f 2024-09-16T00:13:16.999 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:16 smithi102 systemd[1]: Stopping Ceph mds.f for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:17.370 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: 2024-09-16T00:13:16.998+0000 7fce9a459640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:17.370 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:16 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f[64162]: 2024-09-16T00:13:16.998+0000 7fce9a459640 -1 mds.f *** got signal Terminated *** 2024-09-16T00:13:20.120 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:19 smithi102 podman[71302]: 2024-09-16 00:13:19.856875993 +0000 UTC m=+2.972380397 container died c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T00:13:20.466 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:20 smithi102 podman[71302]: 2024-09-16 00:13:20.207024249 +0000 UTC m=+3.322528696 container cleanup c996b89a561a361fff1b07d858b42a7da0c04fb9f6960ee1259a4e830761743c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T00:13:20.467 INFO:journalctl@ceph.mds.f.smithi102.stdout:Sep 16 00:13:20 smithi102 bash[71302]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-f 2024-09-16T00:13:20.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T00:13:20.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:20.675 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T00:13:20.675 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T00:13:20.676 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.i 2024-09-16T00:13:21.011 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:20 smithi102 systemd[1]: Stopping Ceph mds.i for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:21.370 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i[64765]: 2024-09-16T00:13:21.010+0000 7fa9a49d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:21.370 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:21 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i[64765]: 2024-09-16T00:13:21.010+0000 7fa9a49d0640 -1 mds.i *** got signal Terminated *** 2024-09-16T00:13:24.120 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:23 smithi102 podman[71425]: 2024-09-16 00:13:23.851314512 +0000 UTC m=+2.947857703 container died 4531dd4e388b9cdc4e3d0c322fe484bd60a3375d2086a71e8516a0b2a15ae9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T00:13:24.537 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:24 smithi102 podman[71425]: 2024-09-16 00:13:24.269023285 +0000 UTC m=+3.365566466 container cleanup 4531dd4e388b9cdc4e3d0c322fe484bd60a3375d2086a71e8516a0b2a15ae9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:13:24.537 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:24 smithi102 bash[71425]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i 2024-09-16T00:13:24.537 INFO:journalctl@ceph.mds.i.smithi102.stdout:Sep 16 00:13:24 smithi102 podman[71437]: 2024-09-16 00:13:24.420142239 +0000 UTC m=+0.567422918 container remove 4531dd4e388b9cdc4e3d0c322fe484bd60a3375d2086a71e8516a0b2a15ae9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T00:13:24.562 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T00:13:24.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:24.594 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T00:13:24.594 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T00:13:24.595 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.l 2024-09-16T00:13:24.870 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:24 smithi102 systemd[1]: Stopping Ceph mds.l for 21a99a0a-73bd-11ef-bceb-c7b262605968... 2024-09-16T00:13:25.370 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: 2024-09-16T00:13:24.922+0000 7f2f7e8d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:13:25.370 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:24 smithi102 ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l[63861]: 2024-09-16T00:13:24.922+0000 7f2f7e8d8640 -1 mds.l *** got signal Terminated *** 2024-09-16T00:13:28.120 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:27 smithi102 podman[71547]: 2024-09-16 00:13:27.843645507 +0000 UTC m=+3.030135340 container died 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:13:28.665 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:28 smithi102 podman[71547]: 2024-09-16 00:13:28.480866153 +0000 UTC m=+3.667355985 container cleanup 0a7614f79f908ddbced8185e15b034d52da309dc9ce656e97e8be658cd86123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:13:28.665 INFO:journalctl@ceph.mds.l.smithi102.stdout:Sep 16 00:13:28 smithi102 bash[71547]: ceph-21a99a0a-73bd-11ef-bceb-c7b262605968-mds-l 2024-09-16T00:13:28.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-21a99a0a-73bd-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T00:13:28.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:13:28.924 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T00:13:28.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:13:29.137 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:31.718 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:13:31.919 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:34.275 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:36.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:13:36.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:13:36.956 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:13:36.983 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:13:36.984 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi031/crash 2024-09-16T00:13:36.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:13:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:13:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:13:37.028 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi081/crash 2024-09-16T00:13:37.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:13:37.066 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:13:37.067 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:13:37.068 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi102/crash 2024-09-16T00:13:37.069 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:13:37.109 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:13:37.109 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:13:37.110 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:13:37.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-16T00:13:37.150 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:13:37.150 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:13:37.193 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:13:37.196 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:13:37.218 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T00:13:37.218 INFO:teuthology.orchestra.run.smithi081.stderr:: No such file or directory 2024-09-16T00:13:37.218 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:13:37.218 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T00:13:37.218 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:13:37.222 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T00:13:37.222 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T00:13:37.222 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:13:37.222 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T00:13:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T00:13:37.224 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.a.log: 84.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:13:37.224 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:13:37.224 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T00:13:37.227 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:13:37.227 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T00:13:37.231 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log: 87.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:13:37.231 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:: No such file or directory 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip 82.1% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:13:37.233 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr: 90.5% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log: 87.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T00:13:37.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T00:13:37.236 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.4.log: 90.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:13:37.236 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:13:37.236 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T00:13:37.238 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T00:13:37.238 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T00:13:37.238 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:13:37.238 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:13:37.242 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:13:37.242 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:13:37.245 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T00:13:37.245 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-16T00:13:37.246 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T00:13:37.246 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T00:13:37.252 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T00:13:37.253 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:13:37.253 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:13:37.254 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:13:37.258 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:13:37.267 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:13:37.268 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T00:13:37.271 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log: 91.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T00:13:37.271 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T00:13:37.281 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.h.log: 87.4%gzip -5 --verbose -- /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T00:13:37.282 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:13:37.283 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:13:37.284 INFO:teuthology.orchestra.run.smithi102.stderr: 92.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T00:13:37.289 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T00:13:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:13:37.290 INFO:teuthology.orchestra.run.smithi031.stderr: 94.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T00:13:37.297 INFO:teuthology.orchestra.run.smithi031.stderr: 94.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T00:13:37.305 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:13:37.307 INFO:teuthology.orchestra.run.smithi102.stderr: 94.5% 92.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T00:13:37.307 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T00:13:37.318 INFO:teuthology.orchestra.run.smithi102.stderr: 92.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T00:13:37.362 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T00:13:37.373 INFO:teuthology.orchestra.run.smithi081.stderr: 93.0% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T00:13:37.507 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T00:13:37.565 INFO:teuthology.orchestra.run.smithi081.stderr: 93.0% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T00:13:37.637 INFO:teuthology.orchestra.run.smithi102.stderr: 92.0% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T00:13:37.711 INFO:teuthology.orchestra.run.smithi081.stderr: 92.0% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T00:13:38.359 INFO:teuthology.orchestra.run.smithi031.stderr: 90.4% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T00:13:40.221 INFO:teuthology.orchestra.run.smithi102.stderr: 93.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T00:13:40.350 INFO:teuthology.orchestra.run.smithi031.stderr: 93.5% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T00:13:40.695 INFO:teuthology.orchestra.run.smithi102.stderr: 93.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T00:13:40.780 INFO:teuthology.orchestra.run.smithi102.stderr: 93.7% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:13:40.986 INFO:teuthology.orchestra.run.smithi081.stderr: 93.9% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:13:41.042 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T00:13:41.121 INFO:teuthology.orchestra.run.smithi031.stderr: 93.7% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T00:13:41.140 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T00:13:41.205 INFO:teuthology.orchestra.run.smithi102.stderr: 93.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:13:41.207 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T00:13:41.207 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m3.997s 2024-09-16T00:13:41.207 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m14.176s 2024-09-16T00:13:41.207 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.517s 2024-09-16T00:13:41.551 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:13:41.731 INFO:teuthology.orchestra.run.smithi081.stderr: 93.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:13:41.733 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-09-16T00:13:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m4.526s 2024-09-16T00:13:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m16.383s 2024-09-16T00:13:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m0.602s 2024-09-16T00:13:42.497 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/21a99a0a-73bd-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:13:46.211 INFO:teuthology.orchestra.run.smithi031.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-16T00:13:46.213 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T00:13:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m9.005s 2024-09-16T00:13:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m26.191s 2024-09-16T00:13:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.909s 2024-09-16T00:13:46.213 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:13:46.214 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi031/log 2024-09-16T00:13:46.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:13:49.336 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi081/log 2024-09-16T00:13:49.340 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:13:52.602 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi102/log 2024-09-16T00:13:52.611 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:13:54.724 INFO:tasks.cephadm:Removing cluster... 2024-09-16T00:13:54.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force 2024-09-16T00:13:54.938 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:55.224 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force 2024-09-16T00:13:55.427 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:55.778 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 21a99a0a-73bd-11ef-bceb-c7b262605968 --force 2024-09-16T00:13:55.977 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 21a99a0a-73bd-11ef-bceb-c7b262605968 2024-09-16T00:13:56.315 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T00:13:56.315 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:13:56.335 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:13:56.352 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:13:56.369 INFO:tasks.cephadm:Teardown complete 2024-09-16T00:13:56.369 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:13:56.380 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:13:56.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:13:56.383 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:13:56.395 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:13:56.505 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:13:56.515 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:13:56.516 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:13:56.519 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:13:56.522 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:13:56.534 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:13:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:13:56.535 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:13:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:13:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-16T00:13:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +270us[ +147us] +/- 51ms 2024-09-16T00:13:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1055us[-1179us] +/- 41ms 2024-09-16T00:13:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1341us[+1217us] +/- 72ms 2024-09-16T00:13:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:56.540 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:13:56.540 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T00:13:56.540 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +137us[ +145us] +/- 51ms 2024-09-16T00:13:56.540 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1192us[-1192us] +/- 41ms 2024-09-16T00:13:56.540 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +1195us[+1204us] +/- 72ms 2024-09-16T00:13:56.541 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:56.541 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:13:56.542 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-16T00:13:56.542 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +265us[ +133us] +/- 51ms 2024-09-16T00:13:56.542 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1055us[-1186us] +/- 41ms 2024-09-16T00:13:56.542 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +1338us[+1206us] +/- 73ms 2024-09-16T00:13:56.542 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:56.542 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:13:56.596 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:13:56.597 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:13:56.607 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:13:56.691 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:13:56.762 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:13:56.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:13:56.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:56.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:13:56.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:56.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:13:56.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:56.919 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:13:56.933 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:13:56.943 INFO:teuthology.task.internal:Duration was 1806.341215 seconds 2024-09-16T00:13:56.944 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:13:56.953 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:13:56.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:13:56.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:13:56.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:13:56.996 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:13:56.998 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:13:56.999 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:13:57.340 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:13:57.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T00:13:57.340 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:13:57.364 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-09-16T00:13:57.364 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:13:57.402 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T00:13:57.402 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:13:57.423 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:13:57.423 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:13:57.426 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:13:57.444 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:13:57.489 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:13:57.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:13:57.496 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:13:57.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:13:58.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:13:58.033 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:13:58.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:13:58.060 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:13:58.086 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:13:58.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:13:58.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.129 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.147 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-16T00:13:58.150 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-09-16T00:13:58.176 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-16T00:13:58.201 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:58.216 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:58.256 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:13:58.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:13:58.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:13:58.282 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:13:58.283 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi031 2024-09-16T00:13:58.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:13:58.327 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi081 2024-09-16T00:13:58.327 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:13:58.361 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906345/remote/smithi102 2024-09-16T00:13:58.362 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:13:58.398 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:13:58.398 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:13:58.401 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:13:58.403 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:13:58.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:13:58.461 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:13:58.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:13:58.494 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:13:58.495 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:13:58.497 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:13:58.499 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:13:58.510 INFO:teuthology.orchestra.run.smithi031.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:13 /home/ubuntu/cephtest 2024-09-16T00:13:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:13 /home/ubuntu/cephtest 2024-09-16T00:13:58.514 INFO:teuthology.orchestra.run.smithi102.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:13 /home/ubuntu/cephtest 2024-09-16T00:13:58.515 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:13:58.537 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:13:58.579 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} duration: 1806.3412153720856 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:13:58.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:13:58.666 INFO:teuthology.run:pass